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ABSTRACT 


This thesis examines current metrics used by a Marines Aviation Logistics 
Squadron (MALS) and how a MALS can be measured for overall performance in regard 
to supporting flight-line squadrons. Currently, used primary metrics are analyzed for 
their ability to reflect the level of support a MALS provides, the behavior that those 
metrics incentivize, metrics that should be adopted, and how standardized metrics can 
be used to compare performance between various MALS. Supply shelf items with a 
critically low physical buffer status, supply chain response time, and supply 
effectiveness for high-priority parts are found to be the best metrics for overall 
performance measurement. These metrics, when measured specifically according to the 
various type/model/series of aircraft a MALS supports, constitute a performance 
measurement system that can be used by aviation logistics leaders to compare various 
MALS. 
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I. INTRODUCTION 

A. GENERAL OVERVIEW 

A Marine Aviation Logistics Squadron (MALS) provides intermediate 
maintenance and supply support for flight-line squadrons. A MALS might support flight¬ 
line squadrons that all operate the same type/model/series (T/M/S) aircraft or squadrons 
with different T/M/S aircraft. Each T/M/S has its own support network within the greater 
aviation logistics (AVLOG) community. 

Determining the level or quality of support that a MALS is providing its flight¬ 
line squadrons is quite difficult. This is because a MALS performs so many different 
types of support functions that knowing what should be measured can be unclear. Each of 
these support functions also consists of many other sub-functions, cooperating to produce 
a final output, whether that output is a part or a service. 

The civilian sector utilizes many different methods of measuring performance in 
its organizations and several are universally agreed upon as best business practices. 
Applying these performance measurement systems directly to a MALS is difficult for 
many reasons but primarily because financial measurements are a large part of these 
civilian measurement systems. Lurthermore, the supplier-customer relationship in the 
civilian sector is not identical to the MALS-flight line relationship in AVLOG. 

This thesis addresses these issues, and answers three primary questions and one 
secondary question. 

1. Primary Questions 

• Of the metrics that MALS currently employs, which measures (or 
combination thereof) most appropriately reflect the level of support the 
MALS provides the flight-line? 

• Of the metrics that MALS currently employs, which measures most 
appropriately incentivize the most beneficial behavior to support the 
flight-line? 

• Are there metrics that MALS does not employ that should be adopted? 
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2. Secondary Question 

• To what extent can standardized performance measurements be used to 
measure flight-line support performance between various MALS? 

Many of the logistic metrics currently in use measure their respective processes 
sufficiently. These individual metrics, however, are unable to function as a performance 
measurement for the MALS as a whole. This is because they do not sufficiently measure 
the output of the MALS as a whole. A proper performance measurement system for a 
logistics organization will capture the output of that organization, where the output is a 
function of all the sub-functions within the organization. While a MALS has numerous 
and varied sub-functions and responsibilities, only those that directly contribute to the 
goal of the MALS are to be included in the performance measurement system. 

Supply Effectiveness is found to be, of the metrics currently used, the metric that 
most appropriately reflects the level of support the MALS provides the flight-line. 
Repairable Physical Buffer Status Red and TRR-Supply are currently used metrics that 
incentivize the most appropriate behavior to support the flight-line. Supply Chain 
Response Time is an individual metric that should be adopted by the MALS that is not 
currently used. A weighted performance metric composed of three sub-metrics (Physical 
Buffer Status Red of Repairables, Supply Chain Response Time, and Supply 
Effectiveness) should be adopted as an overall performance system that unifies the 
command towards a common goal. Easily, these metrics ought to be aircraft 
type/model/series specific in order to make comparisons of performance between various 
MAES. 
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II. BACKGROUND 


A. INTRODUCTION 

This chapter focuses on the current structure, organization, and measurement 
systems used in a Marine Aviation Logistics Squadron (MALS). The organizational 
structure of a MALS discussed here is specific to the current structure of MALS in 
garrison settings. The systems discussed are the various systems accessible to a MALS 
and are used to measure, report, analyze, or otherwise manage data. Any logistics or 
performance metrics used within a MALS are generally computed by these systems. 

B. ORGANIZATIONAL STRUCTURE OF A MARINE AVIATION 

LOGISTICS SQUADRON 

A MALS exists to provide aviation logistics (AVLOG) support of intermediate 
level maintenance and supply functions to flight line squadrons. A typical MALS consists 
of five departments: Maintenance, Supply, Avionics, Ordnance, and Headquarters. Due 
to their similarities and relationship, the Maintenance, Avionics, and Ordnance 
departments are collectively referred to as the Maintenance department in this thesis. 

A MALS may support flight line squadrons that all fly the same type/model/series 
(T/M/S) aircraft but, more commonly, a MALS will support multiple squadrons with 
different T/M/S. This is depicted in Figure 1. MALS-24 in Hawaii, for example, supports 
one CH-53E squadron, one H-1 squadron that flies AH-IW and UH-IY aircraft, four 
Navy P-3 squadrons (part of Commander, Patrol, and Reconnaissance Wing Two), and 
one H-60 squadron. Each squadron’s aircraft T/M/S is different from the others (with the 
exception of the P-3 squadrons). In practical terms, this means the supply and 
maintenance support required from the supporting MAES becomes more cumbersome 
and complex with each different T/M/S supported. 
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Figure 1. MALS Support Organization Example. Source: United States Marine Corps 

( 2012 ). 


The Aircraft Maintenance Officer, typically an 0-4 limited duty officer, leads the 
Maintenance Department. According to the Marine Corps Warfighting Publication 
3021.2, “the AVLOG functions of the MALS maintenance department include aircraft, 
avionics, ground support equipment (support equipment) maintenance, flight equipment, 
cryogenics, and maintenance data collection and analysis” (United States Marine Corps, 
2012, p. 3-1). Each of these departments is broken down into divisions with each 
focusing on one major function. The Power Plants 400 division, for example, primarily 
focuses on engine overhauls. The Ground Support Equipment 800 division repairs and 
maintains large support equipment for use by the MALS and flight line squadrons. 

The Aviation Supply Officer, typically an 0-4 unrestricted officer, leads the 
Supply Department. The supply department “executes all storage, inventory, condition, 
and management functions of Navy-provided, aeronautical-related materiel” (United 
States Marine Corps, 2012, p. 3-4). For example, the Supply Response Division (SRD) is 
responsible “for the initial screening and technical research of all requisitions ordered 
through NALCOMIS” (United States Marine Corps, 2012, p. 3-4). 
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C. MALSP MODERNIZATION 

A MALS is part of the larger Marine Aviation Logistics Support Program 
(MALSP). The MALSP framework was recently reconstructed and retitled MALSP 
Modernization, or MALSP MOD. MALSP MOD is “an expeditionary demand-pull 
logistics capability solution that aligns with national defense and security strategies, as 
well as: the long war concept. Marine Corps vision and strategy 2025, the maritime 
strategy, the aviation plan, and the aviation logistics (AVLOG) strategy” (Headquarters 
Marine Corps, 2016, p. 1). 

Historically, MALSP functioned as a push supply system. A push supply system 
supports the end user by pushing resources forward to the user, regardless of whether 
requisitions for those resources have been made. This involves a large footprint of supply 
inventory and repair capability in forward deployed areas. An entire MALS would be 
moved to forward areas, providing intermediate level support to other deployed 
squadrons. 

MALSP MOD seeks to create a pull supply system. A pull system’s action is 
dictated by the demand signal produced by the customer. The paradigms on which 
MALSP and MALSP MOD are based are quite different. Rather than a supplier pushing 
parts to fill shelf space, parts are pulled by the customer to fill a demand. Rather than a 
large supply footprint forward deployed, a parent MALS (PMALS) supports deployed 
squadrons from garrison locations, as depicted in Figure 2. These PMALS send 
requisitioned parts forward according to a flight line squadron’s demand signal. The 
logistics chain is established along various nodes between the PMALS and the deployed 
squadrons. Navy T-AVB ships. Main Operating Bases (MOBs), and Forward Operating 
Bases (FOBs) are used to link the PMALS with the squadrons. 
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Figure 2. MALSP II Nodal Network. Source: Borrelli (2011). 


A key aspect of MALSP MOD is the updated technology suite titled Marine 
Aviation Logistics Enterprise Information Technology (MAL-EIT) which will 
accompany the new program. With the intent of being a net-centric logistics chain, the 
next generation buffer management system (NGBMS), expeditionary pack-up kit 
(EPUK), and logistics planning tool (EPT) have been created to better support all MAES 
and squadrons in Marine aviation. These software packages, in combination with the 
nodal logistics chain, “provide more responsive, scalable, and flexible solutions through 
properly-sized spares packages while achieving near real time global visibility” 
(Headquarters Marine Corps, 2016, p. 2). 

D. BUFFER MANAGEMENT TOOL 

The current buffer management tool (BMT) is the primary software package 
designed for use by the maintenance and supply departments to prioritize repair and 
replenishment requirements. Division officers in the MAES are instructed to “use BMT 
reports (I-level) to monitor daily workload and assign priorities to ensure efficient 
movement of components through assigned work centers” (Department of the Navy, 
2012, p. 3-92). The BMT is described in the Naval Aviation Maintenance Program 
(NAMP) as follows: 
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[The BMT] pulls data from NALCOMIS and R-Supply databases to 
generate integrated reports that provide focus on what work has priority 
for repair and replenishment of buffers in a Time Domain. Monitoring 
consumable and repairable components, from a time domain perspective, 
gives supervisors the information they require to ensure they are working 
on the items that are a priority to the customer producing aircraft ready for 
tasking and not sub-optimizing resources by working on lower priority 
items. (Department of the Navy, 2012, p. A-9) 

The BMT provides maintenance and supply managers with the requisite 
information to prioritize their work. It can produce a wealth of reports useful to 
maintenance managers, such as the expeditious repair (EXREP) report, work load report, 
and back log report. Other reports help managers identify what the next EXREP might be 
due to their buffer status as well as identify how the MAES may reduce costs caused by 
being unable to repair components in-house (i.e., causing a repairable to be considered 
beyond the capability of maintenance, or BCM). The supply BMT “provides activities 
with a means of monitoring and managing the time to reliably replenish (TRR) of all 
components under TRR Management. The TRR value of a requisitioned part starts from 
the time a requisition is ordered and ends when the requisition is completed” (NAVAIR 
6.7.2.1,2013,p. G-5). 

The BMT prioritizes work according to its buffer status. The buffer status is 
determined by the item’s TRR. The TRR is defined as “the total time it takes once a part 
is pulled from the Supply shelf until it is back on the Supply shelf ready for issue” 
(NAVAIR 6.7.2.1, 2013, p. G-30). To compute the TRR, the times to fill historical 
requisitions of a particular item are measured. The 90th percentile of those times is then 
applied as the TRR for that part (Seagren, 2013). 

Each requisition has a color code assigned based on its buffer status. The buffer 
status has both a time and physical dimension. In regard to time, the code refers to how 
long it will take to repair the item and get it back on the supply officer’s shelf ready for 
issue. Physical buffer status is characterized by how many items are on the shelf. The 
order of prioritization of work orders, in descending order of importance, is EXREPs, 
red, yellow, green, and black. Eigure 3 is an example workload priority decision matrix 
demonstrating the interaction of the time and physical components of TRR. 


7 



TRR “Rules of Engagement” 

EXKEP aKvay« gat* top priority t 

work'*^od*‘TRR first (fr«tin-rrst sorvod) 

{ \ Work “’Yelkaw* TRR when at stoppage on red 

1 J Wortt "Oreen" TRR after stoppage on yelow and red 

Work'3lack~ into rotation after red-yellow-green(Pr1ornizeTRR Btack items wHh physical buffer needs I.e. SUPPO's sttelf) 






Excess 

111 

Ill 

111 

/// 


n/ HI 

II 

II 

! 1 

Yellow 

1 

1 

1 



Gfte»n 

'Mm 1 

Yellow 

1 TRR 


Black 

TRR 



PARTS 
V BUFFER 
r (Physical 
Inventory) 


SUPPO’S 

SHELF 


V. 


TIME <TRR) BUFFER 


Figure 3. Example BMT Workload Priority Decision Matrix. 

Source: NAVAIR 6.7.2.1 (2013). 


The Maintenance and Supply departments are able to extract a multitude of 
reports analyzing their current workload. Examples of the broad view of Supply 
Department’s workload in terms of Requisition TRR, Consumable TRR, Repairable 
TRR, and Physical Buffer Status are provided in Eigures 4, 5, 6, and 7. 


Supply Buffer Management Tool 
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Eigure 4. Supply Requisition TRR Summary. Adapted from NAVAIR (2011). 
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Figure 5. Supply Consumable TRR Summary. Adapted from NAVAIR (2011). 
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Figure 6. Supply Repairable TRR Summary. 

Adapted from NAVAIR (2011). 
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Figure 7. Supply Physical Buffer Summary. Adapted from NAVAIR (2011). 


The Maintenance Department’s reports are formatted similarly. The Department 
Summary examines backlog and physical buffer impact of inductions, as seen in Figure 
8. Figure 9 is an example Awaiting Parts (AWP) report that summarizes how AWP 
MAFs are affecting the department’s workload. 
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Figure 8. Maintenance Department Summary. Adapted from NAVAIR (2011). 
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Maintenance Buffer Management Tool 
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Figure 9. Maintenance AWP Impact Summary. 

Adapted from NAVAIR (2011). 


E. NALCOMIS 

The Naval Aviation Logistics Command Management Information System 
(NALCOMIS) is the software program that manages maintenance activity in both 
Organizational and Intermediate level maintenance. The NALCOMIS system has two 
different versions, one optimized for the Intermediate level (I-Level) maintenance and 
one optimized for the Organizational level (0-Level) maintenance (primarily 
incorporating maintenance action forms linked to specific flight line aircraft). 

NALCOMIS is a tool intended to help maintenance managers prioritize and 
manage their workload. There are two primary reports that are built into NALCOMIS at 
the I-Level. They are the Daily Production Report and the Production Report. 

The NAMP states that the Daily Production Report “should be run and distributed 
on a daily basis. The Daily Production Report Part 2 provides a count of all maintenance 
actions accomplished from the begin date/time to the end date/time as selected by the 
user” (Department of the Navy, 2012, p. 14-19). This Daily Production Report is an 
overview of maintenance actions that have been completed. Managers can use this 
information to examine the current status of any Maintenance Action Form (MAF) for 
their work center or division. 
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The NAMP states that the Production Report should also be run on a daily basis. 
This report is more detailed, providing more detail about work stoppages due to awaiting 
parts (AWP) or if a part is deemed beyond the capability of maintenance that the work 
center can provide (BCM). This report “provides a snapshot of each work center’s 
production with the number of components and status in the maintenance cycle 
(including OH, AWP, RFI, BCM, and backlog) within a specified begin and end date. A 
percentage of RFI/BCM rate is computed, showing work load production by work 
center” (Department of the Navy, 2012, p. 14-19). 

F. R-SUPPLY 

R-Supply is Aviation Supply’s primary software program that provides online 
inventory, logistics, and financial management tools to the Aviation Supply Department. 
A wealth of data is contained in this program. While various measurements can be 
obtained on every part and requisition in the supply system, the performance metrics 
most used by the Aviation Supply Department are Net Supply Effectiveness and Gross 
Supply Effectiveness. 

Net Supply Effectiveness is the proportion of requisitions filled of items that are 
carried on the supply shelf (Seagren, 2013). If a requisition is placed for an item that is 
not carried on the Supply Officer’s shelf then it is not considered in this equation. Gross 
Supply Effectiveness, however, includes those items that are not carried on the supply 
shelf and is thus the proportion of requisitions filled to total requisitions (2013). 

R-Supply produces two reports that contain these metrics as well as others: the 
Depth Effectiveness report and the Eogistics Replenishment (EOGREP) Supply 
Effectiveness report. The Depth Effectiveness “produces a summary of the total customer 
demands, total issues, and the effectiveness expressed in a percentage” (NAVSUP P-732, 
2005, p. 48). The EOGREP Supply Effectiveness report “produces a report that measures 
the supplying activity’s ability to fill customers’ requisitions and referrals (frequencies). 
The report summarizes the total number of customer demands, total issues, and the 
effectiveness expressed as a percentage by cognizance symbol within MIC” (NAVSUP 
P-732, 2005, p. 48). 
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G. SUPPLY RESPONSE METRICS 

The NAMP describes several basic concepts and guidelines for I-Level 
maintenance in regard to material management. These include having standardized 
requisitioning procedures, maintaining positive control of all accountable material, 
maximizing use of personnel and material resources, and ensuring supply response to 
material demands is optimum (Department of the Navy, 2012). Ensuring supply response 
to material demands is particularly important in the discussion of performance metrics. 

The NAMP dictates the response time goals the Supply Department should strive 
to achieve, determined by the priority of the part requisitioned. Priorities 1, 2, and 3 are 
given to all requisitions with Priority 1 being the most important. It is assigned to parts 
that are required for maintenance on an aircraft that is non-mission capable. Specifically, 
the absence of these parts renders an aircraft non-mission capable. The response time 
goals for these aircraft are as follows: 


Table 1. Supply Response Time Goals 


Issue Priority 
Group 

Priority 

Designator 

Processing 

Time 

1 

1-3 

1 Hour 

2 

4-8 

2 Hours 

3 

9-15 

24 Hours 


It is primarily the Aviation Supply Officer’s responsibility to ensure response 
times are being measured and reduced (Department of the Navy, 2012, p. 9-17). Each 
priority group has a goal processing time in which the requisition is to be filled and 
delivered to the squadron. 

The NAMP states that one of the primary goals of the maintenance and supply 
personnel at all echelons is to reduce response time. It goes on to state that “the 
[turnaround time] of repairables must be improved through better control and reporting 
procedures” (Department of the Navy, 2012, p. 5-28). To monitor this performance, a 


13 




supply response time (SRT) metric is taken that indicates what percentage of requisitions 
are delivered within their target time. 

H. AIRSPEED 

The Naval Aviation Enterprise (NAE) officially implemented a program called 
AIRSpeed in pursuit of increasing fleet readiness in a cost-wise manner. The overarching 
methodologies of AIRSpeed are the Theory of Constraints, Eean Manufacturing, and Six 
Sigma. AIRSpeed is Naval Aviation’s catalyst for continuous process improvement (CPI) 
within aircraft maintenance and aviation supply functions. It “provides a structured 
approach to plan, train, integrate, sustain, and monitor best business practices across the 
NAE. Eunctions include benchmarking, analysis, innovation, progress assessment, 
communications, and incorporation of best practices to maximize overall benefits” 
(Department of the Navy, 2012, p. 1-7). The NAMP identifies six fundamental principles 
of the AIRSpeed program (p. 1-7): 

• Reduce Total Ownership Cost 

• Manage aviation maintenance practices to maximize aircraft availability 

• Identify and reduce non value added process steps (waste) 

• Manage inventory and investments (parts, equipment, and facilities) 

• Manage and reduce the variability in processes 

• Create a culture of CPI 

Each MAES has a dedicated AIRSpeed office. This office is designed to be led by 
an AIRSpeed officer with a minimum of three other personnel. This team will consist of 
at least two Green Belt-certified personnel and one Black Belt-trained individual. 
Eurthermore, each division within the MAES is to have two Green Belt certified 
personnel. 

The AIRSpeed office offers Yellow Belt and Green Belt Eean Six Sigma training. 
The trainees are then encouraged to return to their divisions to implement process 
improvement events. Identification of projects can happen in various ways. Most often. 
Marines recognize areas for improvement in their own divisions simply by being subject 
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matter experts and knowing what is and is not working. They experience the constraints 
on a day-to-day basis. The AMO has a high-level view of the maintenance department 
and is able to identify problems that affect multiple divisions or departments. The 
enterprise project alignment tool (EPAT) is another way that projects are identified. 

The EPAT is used by an AIRSpeed office to focus its efforts. It “translates 
strategic level objectives to tactical goals and tasks” (NAVAIR 6.7.2.1, 2013, p. G-12). 
Within the EPAT, there are three primary documents: goal alignment chart (GAC), site 
opportunity worksheet (SOW), and tracking matrix (TM). The example EPAT in Eigure 
10 demonstrates the interaction of various organizations’ goal and objectives. 

The GAC is a command-level document that identifies process improvement 
opportunities that are aligned with strategic goals. The SOW is the AIRSpeed officer’s 
tool for tracking projects that will help meet the strategic goals. The TM is the tool that 
portrays project progress as well as how the AIRSpeed projects are aligned with NAE 
objectives, command goals, and department goals. 


Dept. 

Goals 




Site Oppoi tunin' 
Worksheet 

AIRSpeed 

Projects 

1. RIE Event m 62 V 

2. BE Project 9 

3. RIEm Medical 

4. VSA in 440 

5. RIEm Eng. 

r 


N.4E 

Objectives 


C oiiimaiid 
Goals 


Enable Navy & Marine Corps to meet 
core capabilities 


Eigure 10. Example Project Alignment Tool 
Source: NAVAIR 6.7.2.1 (2013). 
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I. 


CURRENT READINESS 


The Marine Corps’ Current Readiness program is a division of the Naval Aviation 
Enterprise (NAE) tasked to “to improve the delivery of combat ready forces to meet 
current and future operational requirements at an optimal Operating and Support (O&S) 
cost” (Naval Aviation Enterprise, n.d.). The Current Readiness program spans the 
entire system of Naval Aviation. The metrics used in this program inform Naval Aviation 
leaders of the current standing of a T/M/S fleet in terms of readiness, performance, 
and competency. 

Some of the key metrics used are Aircrew Core Competency, Maintainer Core 
Competency, Aircraft Availability, and Cost per Elight Hour. These metrics are separated 
per T/M/S and examined at a “per squadron” level as well as a “total fleet” level. All of 
these metrics are in relation to a respective achievement target. 

The Current Readiness program also examines cost and schedule performance of 
the T/M/S fleet, using popular Earned Value Management principles. The cost 
performance index is calculated based on planned versus actual total cost of contracts, 
fuel, repairables, and consumables. The schedule performance index is calculated based 
on the flight hours programmed versus actually flown for a given fiscal year. The actual 
cost in relation to the actual executed flight hours provides the Current Readiness with an 
Execution Index. 

If the Execution Index is above or below a certain threshold, leaders dig deeper 
into the root cause of the variation. A cycle of briefs occurs throughout the year that 
spans the entire chain of command in Naval Aviation. These briefs are a collaborative 
effort of all levels in Navy and Marine aviation to address the root cause of problems that 
are negatively affecting readiness or costs in the fleet. 

J. DOD SUPPLY CHAIN IMPLEMENTATION GUIDE 

In 2000, the Logistics Management Institute (EMI) developed the Department of 

Defense (DOD) Supply Chain Management Implementation Guide. This report 

acknowledges “the lack of correct and comprehensive supply chain metrics in DOD” 

(Logistics Management Institute, 2000). It goes on to suggest that many of the metrics 
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used in supply chain management are financial or accounting measures inappropriately 
applied to supply chain manamagement. This is evidenced by “a consensus in DOD [that] 
considers the metrics currently available to senior DOD managers to be inadequate or 
lacking the depth to measure effectivenes and efficiency across the DOD supply chain” 
(Logistics Management Institute, 2000, p. 30). LMI suggests, therefore, that supply 
chains be measured on the basis of ten service quality factors: tangibles, reliability, 
responsiveness, competence, courtesy, credibility, security, access, communication, and 
understanding the customer. 

Balestreri and McDoniel apply LMI’s strategy to supply chain management to a 
MALS by adapting the Supply-Chain Operations Reference model (SCOR), a 
performance measurement system popular in civilian industry. They defined the overall 
goal of AVLOG to be aircraft readiness (Balestreri & McDoniel, 2002). A MALS is only 
one factor of many that can enable a flight line squadron to increase aircraft readiness, 
however. Therefore, they conclude that inventory management and production 
management are the “primary critical success factors” for contributing to aircraft 
readiness and “logistics responsiveness and material availability [are] key conditions for 
achieving success” (Balestreri & McDoniel, 2002, p.83). 

Logistics Responsiveness was simply expeditiously delivering parts to the 
customer when requested, and material availability was comprised of 1) 
ensuring at least one part was available for issue off the shelf and 2) 
reducing DIFM material (due-in-from-maintenance 84 or work in process 
inventory)... These concepts follow the adapted SCOR logistic 
performance metrics of Supply Chain Response Time and Perfect Order 
Fulfillment. (Balestreri & McDoniel, 2002, p. 84) 

They state that logistics response time (LRT) and customer wait time (CWT) are 
“the most significant metrics the MALS can utilize to improve aircraft Ao with respect to 
responsiveness” (p. 87). Balestreri and McDoniel define LRT for the MALS as “the total 
average time spent on-station from initiation of a squadron’s requisition to the delivery 
and completion of that requisition,” and they define CWT as “the total average [time] 
spent processing customer’s requisition on-station and ultimately satisfying that 
requisition from off-station or upstream members of the Naval/Marine Corps aviation 

logistics supply chain” (p. 86). Succinctly, LRT is the amount of time a requisition 
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spends on-station and CWT is the time spent off-station. The combination of these times 
is the Supply Chain Response Time. 

An additional metric they emphasize is perfect order fulfillment, defined as “the 
ratio of perfectly satisfied orders to all orders measured” (p. 93). This is generally 
equivalent to a common fill rate metric used in industry sectors. The metric reflects 
material availability as a success factor in contributing to aircraft readinesss. 

Perfect order fulfillment and supply chain response time are two metrics that, 
when optimized, will contribute to increased aircraft readiness, the goal of AVLOG. 
Balestreri and McDoniel conclude that “perfect order fulfillment and supply chain 
response time must be integrated throughout not only the MATS, but also throughout the 
entire Naval/Marine Corps aviation logistics supply chain. Utilizing these two 
performance metrics throughout the entire enterprise will focus all members of the supply 
chain on the common goal of aircraft readiness” (p. 112). 

K. CHAPTER SUMMARY 

The nodal logistics laydown structure of MALSP MOD requires that all members 
in the logistics chain are working toward the same goal in order to be successful. To 
measure the progress toward that goal, data management systems such as BMT, 
NALCOMIS, and RSUPPLY have been established to document and measure the major 
functions of the MALS. Programs such as AIRSpeed and Current Readiness have been 
established to improve processes and appropriately allocate resources from a systematic 
level. These systems and programs are established to align all of AVLOG to the common 
goal of sustaining aircraft readiness at a high enough level to enable aircrew to achieve 
and maintain appropriate competencies. 
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III. LITERATURE REVIEW 


A. INTRODUCTION 

This chapter discusses current measurement systems that are considered best 
business practices in the industry realm. It also discusses two different approaches to 
performance measurement adopted by the Air Force and the Navy submarine fleet. An 
understanding of these performance measurement systems will aid in the discussion of 
what may and may not be applicable to a MALS. 

A popular adage among military and business professionals states what you 
measure is what you will get. “Inspect what you expect” is a version of this concept often 
repeated in the Marine Corps. Eliyahu Goldratt, who popularized the theory of constraints 
management principle, wrote, “Tell me how you measure me, and I will tell you how I 
will behave” (Goldratt, 1990, p. 26). The link between measurements and performance 
has led to the creation of many measurement systems in both civilian and military 
organizations. This literature review examines logistics and supply chain performance 
measurements and their use among civilian and military organizations. 

B. METRICS AND MEASUREMENT SYSTEMS 

There is a necessary distinction to be made between performance metrics and 
performance measurement systems. Managers may easily make the mistake of assigning 
a single performance metric as their performance measurement system, using an 
individual part to be indicative of the whole. A performance metric, however, is a metric 
used within a performance measurement system and largely focuses on measuring 
individual processes (Caplice & Sheffi, 1994). 

Logistics does not consist of a singular activity, however, and thus a singular 
measurement is insufficient to measure a system’s performance. Logistics “encompasses 
a complex set of activities which require a collection of metrics to adequately measure 
performance” (Caplice & Sheffi, 1995, p. 61). A performance measurement system is 
built upon performance metrics. 
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Not all performance metrics and performance measurement systems are helpful, 
however. Caplice and Sheffi provide separate criteria for evaluating performance metrics 
and performance measurement systems. The eight criteria for evaluating individual 
logistic performance metrics are defined in Table 2. 


Table 2. Eight Performance Metric Evaluation Criteria. 

Adapted from Caplice and Sheffi (1994). 


Performance Metric Evaluation Criteria Summary 

Criteria 

Description 

Validity 

The metric accurately captures the events and activities being 
measured and controls for any exogenous factors. 

Robustness 

The metric is interpreted similarly by the users, is comparable across 
time, location, & organizations, and is repeatable. 

Usefulness 

The metric is readily understandable by the decision maker and 
provides a guide for action to be taken. 

Integration 

The metric includes all relevant aspects of the process and promotes 
coordination across functions and divisions. 

Economy 

The benefits of using the metric outweigh the costs of data 
collection, analysis, and reporting. 

Compatibility 

The metric is compatible with the existing information, material, and 
cash flows and systems in the organization. 

Level of 
Detail 

The metric provides a sufficient degree of granularity or aggregation 
for the user. 

Behavioral 

Soundness 

The metric minimizes incentives for counter-productive acts or 
game-playing and is presented in useful form. 


Not all performance measurement systems are equally helpful, either. The system 
can be “well designed at the strategic level [but] can be flawed at the individual metric 
level” (Caplice & Sheffi, 1994, p. 11). Six evaluation criteria for performance 
measurement systems are shown in Table 3. Crafting performance measurement systems 
according to these criteria will help ensure the resulting actions of those measured by the 
system will best benefit the organization and its customers. 
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Table 3. Six Performance Measurement System Criteria. 

Adapted from Caplice and Sheffi (1995). 


Performance Measurement System Evaluation Criteria Summary 

Criteria 

Description 

Comprehensive 

The measurement system captures all relevant constituencies and 
stakeholders for the process. 

Causally Oriented 

The measurement system tracks those activities and indicators 
that influence future, as well as current, performance. 

Vertically 

Integrated 

The measurement system translates the overall firm strategy to all 
decision makers within the organization and is connected to the 
proper reward system. 

Horizontally 

Integrated 

The measurement system includes all pertinent activities, 
functions, and departments along the process. 

Internally 

Comparable 

The measurement system recognizes and allows for tradeoffs 
between the different dimensions of performance. 

Useful 

The measurement system is readily understandable by the 
decision makers and provides a guide for action to be taken. 


No organization can excel completely in all criteria, whether for performance 
metrics or performance measurement systems. Necessary tradeoffs must be made among 
the metric criteria. The integration and usefulness criteria, for example, face a necessary 
trade-off between scope and span of control (Caplice & Sheffi, 1994). Additionally, as a 
metric becomes more robust, it allows for more comparability between organizations 
or units. However, this negatively impacts its validity as it has to be less specific to 
an organization. 

The goal of a metric is to be indicative of the level of performance of an 
individual process. The goal of a measurement system, however, is to “guide and 
influence the decision making process” (Caplice & Sheffi, 1995, p. 65). In order properly 
affect decision makers, a performance measurement system ought to be developed in 
terms of output and the customer. 

Regarding output, “the basic transaction [for logistics] is a completed delivery to 


a customer” (1995, p. 65). The output (i.e., completed deliveries) and the customer are to 

be the focal points of a performance measurement system because the output is a result of 
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all processes within an organization and the delivery to a customer is the extent to which 
a logistics system has influence. A diagram illustrating how a system’s output is a 
function of individual processes is seen in Figure 11. 


Function Versus Process Output For Logistics 


Outbound Logistics Process 


Function Outputs 


1 

1 

Order Receiving 

1 

1 

1 

1 

Material Handling 

1 

1 

Warehousing 

1 

1 

1 

Transportation 

1 

1 

1 

Customer Service 

1 


# of Purchase Orders Processed 
SKU’s/$ Value Procured 


# of Line Items/Orders Picked, Packed, & Loaded 


# of Items Stored 

Total $ Value of Items Stored 

Tor-Miles Transported 

# of Tons/CWT Shipped 

# of Customer Calls Handled 

# of Invoices/Une Items Entered & Reconciled 


Completed Deliveries 


Process Output 


Figure 11. Process Output as a Product of Function Outputs. 

Source: Caplice and Sheffi (1995). 


C. COMMON PERFORMANCE MEASUREMENT SYSTEMS 

Popular measures of performance in the civilian industry realm include the 
Supply Chain Measurement System (SCM), Balanced Scorecard, Economic Value Added 
(EVA), Process-Based Measurement, and Supply-Chain Operations Reference model 
(SCOR). Companies adopt and adapt these systems to create a customized fit for their 
organization that meets their specific needs. 

Piotrowicz and Cuthbertson (2014) surveyed 51 international companies 
regarding the metrics they use, the factors that influence those metrics, and their 
perceived importance to the organization. These companies described their core activities 
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as manufacturing, retail/wholesaling, transport, logistics value added, warehousing, or 
some combination thereof. Of these companies, the performance measurement systems 
used most often were the Balanced Scorecard (19 companies), Process-Based 
Measurement (15 companies), and SCOR model (12 companies). 

The Balanced Scorecard was developed by Kaplan and Norton (1992) to be “a set 
of measures that gives top managers a fast but comprehensive view of the business” (p. 
71). It provides insight into the performance of an organization through four difference 
perspectives: customer perspective, internal perspective, innovation and learning 
perspective, and financial perspective. As seen in Figure 12, Balanced Scorecard seeks to 
focus an organization on overarching goals and strategy while traditional measurement 
systems focus on individual metrics. Traditional measurement systems may cause 
employees to work to optimize the metric rather than the goal it is meant to serve. The 
Balanced Scorecard focuses the whole organization on a unifying goal. 


Financial Perspective 
Goals and Measures 

✓ \ 


Customer 

Perspective 


Internal Business 
Perspective 

Goals and Measures 


Goals and Measures 


\ ✓ 

Innovation and 
Learning Perspective 

Goals and Measures 


Figure 12. Balanced Scorecard Diagram. Adapted from Balanced Scorecard Institute 

(2015). 
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The Balanced Scorecard is helpful because the organization must define goals not 
only in terms of financial performance (which is easily measured and understood by 
stakeholders) but in other more intangible aspects of a corporation’s performance. By 
setting goals in all four perspectives, a company must also determine measures that 
convey effectiveness in achieving those goals. 

The third primary measurement system respondents used was the Process-Based 
Measurement system. It takes an analytical approach that focuses on the supply chain as a 
process divided into six main sections: supplier, inbound logistics, core manufacturer, 
outbound logistics, marketing and sales, and end customers (Chan & Qi, 2003). Processes 
are decomposed into core processes, sub-processes, and activities, driven by goals and 
defined in terms of responsibility and function, as shown in Figure 13. A primary 
advantage of a process-based measurement system is that it creates opportunity for 
continuous process improvement. The Balanced Scorecard focuses on prior performance 
and is not able to readily identify problems in the supply chain process. Process-based 
measurement utilizes a metric called Performance of Activity (POA). These 
measurements include tangible and intangible aspects of the process and include cost, 
time, capacity, capability, productivity, utilization, and outcome. 


SCM Context 



Figure 13. Process-Based Measurement System. 

Source: Chan and Qi (2003). 
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The SCOR model is also a process focused measurement system but it focuses on 
four main processes: plan, source, make, and deliver. The model continues to break those 
four main categories into two more levels of sub-processes but those four functions are 
the primary focus, as seen in Figure 14. SCOR “spans interactions pertaining to 
customers/markets and transactions pertaining to products” (Khare, Saxsena, & Teeware, 
2012, p. 29). While the SCOR model rated third in performance measurement systems by 
companies in the Piotrowicz study, specific SCOR metrics were used the most 
individually. 


Plan 



Figure 14. The SCOR® Model. Source: Defense 
Acquisition University (2012) 

D. SCHEDULING AND DISPATCH RULES 

Maintenance managers are continually faced with decisions about workload 
scheduling and which items should take priority for repair. Various methods exist, called 
dispatch rules, that aide decision makers in this task. Examples of popular dispatch rules 
include prioritizing jobs by shortest processing time, longest processing time, earliest due 
date, first-come-first-serve, and critical ratio (Baker & Trietsch, 2009, p. 359). 

Dispatch rules are helpful for managers because they enable quick decision 
making for prioritizing jobs. Each time a job is completed, a new priority must be 
determined. If the dispatch rule of shortest processing time is followed, then the 
estimated processing times of all jobs are examined and priority goes to the job with the 
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shortest time. The inverse is true of the longest processing time. The earliest due date 
dispatch rule prioritizes jobs based upon their due date and disregards their estimated 
processing time (2009). A first-come-first-serve dispatch rule prioritizes jobs in the order 
in which they are received. 

The critical ratio dispatch rule is a ratio of remaining allowance and remaining 
work. Baker and Trietsch state, “critical ratio priorities measure urgency by the ratio of 
remaining allowance and remaining work rather than their difference” (p. 359). Those 
with the smallest critical ratio receive priority while a ratio value of 1 indicates a job has 
the most work left to do but also the most time in which to do it. 

E. SERVICE LEVEL 

Service levels and fill rates are common metrics used in supply chain 
management to measure the effectiveness of an inventory management policy. Nahmias 
(2009) titles these metrics as Type 1 Service (also known as Service Level) and Type 2 
Service (also known as Fill Rate). Service level is “the probability of not stocking out in 
the lead time” (p. 272). Fill rate is defined as “the proportion of demands that are met 
from stock” (p. 273). 

The fill rate is a commonly used service metric and is generally what is meant 
when a manager speaks of service level (Nahmias, 2009). This metric is important for 
determining inventory policy in order to ensure a certain level of customer satisfaction is 
met. The cost to provide 100% satisfaction can often be too great for an organization to 
absorb so a certain level of risk must be accepted on behalf of an organization. The risk 
tolerance of an organization is generally determined by the cost of that risk. 

For example, if a shortage cost is extremely high and a holding cost is extremely 
low, then it is easy for an organization to justify a high level of inventory in order to 
optimize customer satisfaction. If shortage costs are low and holding costs are high, it is 
easier to justify accepting more risk for the sake of optimizing financial performance. 
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F. LEAN SIX SIGMA AND THEORY OF CONSTRAINTS 

Lean Six Sigma and the Theory of Constraints are not performance measurement 
systems but they are widely used within logistics and supply chain management. Their 
relationship with performance measurement is quite direct, however. The Theory of 
Constraints focuses on managing the constraints in a process in order to improve 
productivity. Lean methodology seeks to eliminate waste within a process and therefore 
reduce cycle time. Six Sigma is a statistical control process that focuses on reducing 
variation in a process that can be costly. 

To implement any of these process improvement methodologies, one must define 
the goal of the process as well as determine how to measure that goal. Once that goal is 
defined, an organization can begin improving processes that specifically contribute to 
achieving that goal. The DMAIC model of process improvement (Define, Measure, 
Analyze, Improve, and Control) uses these methodologies to optimize a process 
according to a specific goal. One must define tolerances in a process in order to 
implement Six Sigma and one must define an overall measure of performance according 
to a goal in order to implement Theory of Constraints. 

G. AIR FORCE 

The Air Force Logistics Management Agency (AFLMA) and the Air Force 
Materiel Command (AFMC) both developed and recommended separate performance 
measurement systems for use by the Air Force. Leonard (2004) examined these two 
performance measurement systems and found that the AFMC system is modeled around 
four primary metrics: Aircraft Availability, Requirements Computation, Asset Allocation 
and Funding, and Real World Performance. The AFLMA focused rather on four core 
processes: Repair, Buy, Stockage and Distribution, and Funding. 

AFLMA’s performance measurement system espouses a Balanced Scorecard 
approach with six segments, as seen in Figure 15. These six segments have 23 assigned 
metrics in total measuring each segment’s performance (Leonard, 2004). 
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Figure 15. AFLMA’s Balanced Scorecard. 

Source: Leonard (2004). 

The AFMC performance measurement system uses only ten metrics, five 
measuring performance and five measuring process, as depicted in Figure 16. The 
simplicity of this model may assist managers in the decision making process more so than 
AFLMA’s measurement system of 23 metrics. 
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Figure 16. Collective Overview of AFMC’s Measurement System. 

Source: Leonard (2004). 

H. NAVY SUBMARINE FLEET MAINTENANCE METRICS 

The Navy created a Submarine Fleet Maintenance Metric Working Group 
(Working Group) tasked with developing various metrics to measure the performance of 
submarine Fleet Maintenance. The Working Group “took the perspective of viewing 
Fleet Maintenance as a complex system, with multiple stakeholders, executing a series of 
integrated processes” (Submarine Fleet Maintenance Metric Working Group, 2013, p. ii). 

The systems approach enabled the Working Group to create an “integrated set of 
metrics that measure performance, productivity, planning, execution, finance, and 
backlog, while reducing the impact on the already limited bandwidth of those overseeing 
intermediate level maintenance” (Submarine Fleet Maintenance Metric Working Group, 
2013, p. ii). This systemic view and the integration of metrics enables results in 
performance and productivity metrics that are positively and negatively affected by the 
more discrete metrics of which they consist. 
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The Working Group developed two primary metrics, a productivity metric and a 
performance metric. They generally define the productivity metric (PROD) as “output 
divided by input.” It consists of four variables, demonstrated in Equation 1: routine 
maintenance (2K), periodic maintenance requirements (PMR), and alterations (ALTS), 
divided by the total direct cost (CTC) (Submarine Fleet Maintenance Metric Working 
Group, 2013). 

Y.(2K + PMR + ALTS) 

Productivity — -- 

Equation 1. Working Group Productivity Metric 
Source: Submarine Fleet Maintenance Metric Working Group (2013) 

The second primary metric the Working Group developed was the performance 
metric (PERF). It is defined as “the weighted sum of on-time delivery (TD), total direct 
cost (COST), planning event completion (PENG), late work (LW), deferred work (DW), 
and backlog (BE) divided by the sum of the weights” (Submarine Fleet Maintenance 
Metric Working Group, 2013, p. 38). Mathematically, it is defined as: 

.420TZ) + .222COST + .132PLNG + m2LW + .072DW + .072BL 

Performance = -- 

Sf.i 

Equation 2. Working Group Performance Metric 
Source: Submarine Fleet Maintenance Metric Working Group (2013) 

The variables in PERF go beyond typical schedule and cost variables. Other 
variables were added that were considered highly important by the Working Group. The 
basic PERF metric was given several different varieties to better serve the concerns of 
different stakeholders in the system. For example, activities within the logistic system are 
affected differently by late work and therefore the PERF metric weights that variable 
differently to better capture the magnitudes by which late work affects their performance 
individually. The basic PERF metric format, however, is largely the same for each 
activity. The weights for each variable are assigned according to the opinions of the 
subject matter experts in the Working Group. 
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I. CHAPTER SUMMARY 

Logistics metrics and performance measurement systems are not equivalent. For 
either of them to be useful to decision makers, they ought to sufficiently meet the criteria 
proposed by Caplice and Sheffi (1994). In recognition of the delineation between 
individual metrics and measurement systems, civilian industry has adopted several 
performance measurement systems. These systems provide holistic solutions to measure 
the key components of a business that contribute most directly to an organization’s goal. 

These systems, however, largely consist of certain financial metrics that are not 
directly applicable to many military organizations. Given these shortcomings, the Navy 
submarine maintenance fleet, for example, created a performance metric that met their 
specific needs. This performance metric assigns weights to key logistic metrics and 
reports the weighted average as a holistic measurement of performance. 
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IV. ANALYSIS 


A. INTRODUCTION 

How one is measured influences how one behaves. This underlying idea is the 
foundation of this section. An organization must choose carefully which measurements it 
uses to determine how it is performing. These measurements, when emphasized, will 
influence an individual’s behavior, the performance of an organization, and the strategy 
an organization adopts. This analysis section examines primary metrics currently used by 
a MALS and how they affect individuals, the organization, and the customer. 

B. DEFINING THE GOAL OF A MALS 

Balestreri and McDoniel (2002) may very well be correct in concluding that 
aircraft readiness is the goal of aviation logistics as a whole. However, defining this as 
the goal of a MALS, specifically, is problematic. There is an important disconnect 
between a MALS and aircraft readiness that must be recognized. 

The performance of a MALS may directly contribute to aircraft readiness but it 
cannot solely cause aircraft readiness, though a well-perforing MALS is necessary for 
high aircraft readiness. If a MALS can solely cause aircraft readiness then an optimized, 
perfectly performing MALS in ideal conditions would result in perfect aircraft readiness 
among the flightline squadrons it supports. This, however, is not the case. An optimized 
MALS will result in RFI parts being delivered to flight line squadrons as soon as they are 
needed by those squadrons, enabling though squadrons to achieve aircraft readiness. That 
is the extent of the output of a MALS. 

MALS certainly performs more functions for flight-line squadrons than delivering 
RFI parts. MALS Marines may conduct nondestructive inspections (NDI) of helicopter 
blades or other airframe components. They may help troubleshoot low power on an 
engine. There are many functions a MALS serves for their flight line squadrons but the 
vast majority of them, in some sense, result in an RFI part being delivered for a part 
being deemed RFI. Determining a blade is safe for flight through an NDI is a form of 
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rendering the potentially non-RFI blade as, in fact, RFI. Helping troubleshoot low power 
engines and bringing them up to normal power is a form of making that engine RFI. 

The MALS’s extent of influence on readiness in a flight line squadron ends once 
the ready-for-issue (RFI) part is delivered to the flight line squadron. The squadron must 
then install the part, the correctly installed part must actually solve the discrepancy for 
which the part was ordered, and, depending on the level of maintenance performed, the 
mainteners and aircrew may need to successfully perform a functional check flight. Only 
then will that aircraft be able to contribute to an increase aircraft readiness. The goal of 
aviation logistics as a whole may be to increase aircraft readiness but that is beyond the 
scope of a MALS. 

While a MALS has many functions and responsibilities (i.e., maintenance on 
repairable components, inventory management, supply chain management, supporting 
squadron detachments and deployments, training Marines, ensuring the welfare of 
Marines and their families), the ultimate goal of a MALS is to provide ready-for-issue 
(RFI) parts to flight line squadrons. An organization has many responsibilities that do not 
contribute to achieving its goal but those responsibilities must not be misunderstood as 
that organization’s goal. The Marine Corps, for example, has not established a MALS in 
Hawaii in order to train maintenance Marines. That is merely one of its responsibilities. A 
MALS exists in Hawaii to best support the flight line squadrons there with the parts that 
they need. 

The goal of the MALS must not be confused with its various responsibilties. 
Further, the goal of the MALS cannot exceed the end point of the supply chain process 
that it is able to control. That end point is when an RFI part is delivered to the squadron 
or an I-Level maintainer is able to sign off a MAF submitted to the MALS from a 
squadron. The goal of the MALS, therefore, is the output of its functions—specifically, 
RFI parts. 

C. DEFINING A GOOD METRIC 

Currently, a MALS has metrics measuring its many individual functions or 
processes. Identifying an overarching performance measurement system is much more 
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difficult. For example, the Buffer Management Tool, discussed in this chapter, best 
manages pertinent information and measurements of primary functions in a MALS, yet it 
is not necessarily used as a performance measurement system. Rather, individual data 
process measurements from the BMT are most commonly captured and analyzed. 

MALS performance as a whole ought not to be measured solely in terms of TRR, 
BCM rates, or any other individual metric. These metrics may be important for internal 
analysis but they do not reflect the level of success a MALS has in obtaining its goal. For 
example, black MAFs in regard to Design TRR are a maintenance specific metric 
whereas Supply Effectiveness is a supply focused metric (it is affected by maintenance 
performance but that is not the focus of the metric). This does not mean, however, that 
these metrics are to be disregarded. It simply means these metrics are indicative of 
internal process performance only. 

Caplice and Sheffi (1994) do not necessarily distinguish between criteria that 
make good logistics metrics and criteria that make good performance measurement 
systems. The distinction is necessary because the purposes of each are different. A 
logistics metric is internally focused, examining the efficiency or productivity of a 
process. A performance measurement system examines the overall system’s ability to 
produce its output, understood in terms of a product or service for a customer. A 
performance measurement system has broader managerial implications than a logistics 
metric. 

Caplice and Sheffi’s criteria for performance metrics and performance 
measurement systems, summarily given in Table 4, are used as the criteria for this 
evaluation of metrics currently used by MALS to measure performance. A MALS has no 
standard performance measurement system and thus the metric criteria, rather than the 
measurement system criteria, underlie the evaluation of current MALS metrics. 
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Table 4. Summary of Criteria 


Metric Criteria 

Measurement System 
Criteria 

Validity 

Robustness 

Usefulness 

Integration 

Economy 

Compatibility 

Level of Detail 

Behavioral Soundness 

Comprehensive 

Causally Oriented 
Vertically Integrated 
Horizontally Integrated 
Internally Comparable 
Useful 


D. MEASUREMENTS AND PERFORMANCE 

Performance metrics influence the actions of Marines within a MALS in 
particular ways. These actions contribute to a certain level of respective performance for 
the MALS as a whole. This section examines how MALS performance is impacted by 
using these metrics. More specifically, this section examines the resulting effects that 
flight line squadrons experience as a result of optimizing respective metrics. Optimizing 
the metric, in this context, refers to the MALS using its resources to improve the activity 
that the metric measures. 

1. Buffer Management Tool—Buffer Status 

The Buffer Management Tool (BMT) is not a metric in and of itself. It does, 
however, provide a wealth of analytical data that is useful for the user. Many metrics a 
MALS uses are pulled from the BMT. One metric in particular that the BMT produces 
that is used frequently (daily) and widely within the MALS maintenance department is 
Buffer Status. Another is TRR. 

The Time to Reliably Replenish (TRR) measures the amount of time a part has 
spent being repaired in relation to the expected amount of time to repair that part. This is 
primarily understood in this context as a maintenance metric. The second metric is a 
part’s physical buffer status and it refers to the physical inventory level of that part on the 


36 




ASO’s supply shelf. These metrics are often reported in terms of how many black or red 
MAFs there are currently. 

The BMT organizes Maintenance Action Forms (MAFs) by the time projected to 
complete, or sign-off, the MAF. Once a repairable part comes to the MALS for repair, a 
MAF is initiated. This MAF documents the current status of the repair. The status may 
range from being In-Work (IW) to Awaiting Parts (AWP). A myriad of other status codes 
exist but explanation of them does not contribute to the current discussion. 

While the BMT provides a wealth of information regarding the status of each 
MAF and each part on the shelf, certain metrics are used most often to manage the 
workload. For the Supply Department, these include the Requisition TRR, Consumable 
TRR, Repairable TRR, and Physical Buffer Status. These show what is the current TRR 
versus Design TRR of parts as well as how many parts have a physical buffer status of 
red. For the Maintenance Department, these include backlog and physical buffer impact 
of inductions as well as Awaiting Parts (AWP) Impact reports. These show how their 
workload is impacting the physical buffer status and how much of it is due to 
maintenance delays rather than awaiting parts from supply. 

The purpose of the BMT is to help maintenance managers better prioritize their 
workload. According to the NAE CPI Guidebook: 

BMT was initially created to provide Intermediate Maintenance Activities 
(I-Level) the insight necessary to analyze and understand the demand 
placed on their production and to help understand their ability to meet that 
demand while revealing the time it actually takes them to reliably repair 
(or replenish) the asset to ready for issue (RFI). BMT provides the ability 
to see constraints in the production system and to make decisions where to 
best concentrate CPI activities to improve production capability (Time to 
Reliably Replenish or TRR) where it will have the greatest impact. 
(NAVAIR 6.7.2.1, 2013, p. 6-8) 

To accomplish this task, the BMT color codes each MAF according to the 
benchmarked time to repair that part, the TRR. The NAE CPI Guidebook (2013) explains 
how each color code should be prioritized (p. 6-14). Eor example, any MAP with a 
physical buffer status of red should be worked first and in accordance with the TRR. If a 
part has a benchmark of 30 days for repair, it is expected to take 30 days to repair that 
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part. During days 21-30, the MAP has a color code of red. Red MAPs should be worked 
first, in a “first in, first served” manner. During days 11-20, the MAP has a color code of 
yellow. If a MAP is yellow, work should only take place if there is a stoppage on all red 
MAPs. During days 1-10, this MAP has a color code of green. During the green phase, 
work on this repairable should only commence after there is work stoppage on all yellow 
and red MAPs. After day 30, the MAP has a color code of black. Black MAPs, the 
guidebook explains, should only be worked on after red, yellow, and green MAPs are 
unavailable for further work. These black MAPs should be prioritized according to the 
ASO’s physical buffer needs. 

The reliability of the benchmark, or the Design TRR, for each repairable is 
therefore critical to how beneficial the BMT is. The process a MAPS generally takes to 
set repair time benchmarks is quite cumbersome and potentially inaccurate. To set a 
benchmark for a part, a work center within a division will designate a “Design Team” to 
analyze the repair process and determine a reasonable estimation of the repair time for 
that part. To do so, it chooses the repairable part and track the time of every step required 
to complete the repair on that part. That repair time is then set as the Design TRR, or 
benchmark, for that repairable. 

Using a sample size of one cannot produce a reliable benchmark, however. In 
order to mitigate this, work centers will generally deduct from the total time any steps 
that take unusually long. The evaluation of these steps as potential outliers is based on the 
knowledge of subject matter experts on the Design Team. While this may improve the 
reliability of that repair time as representative of population, it still falls short of 
providing a reliable sample. 

To further cloud the problem, parts are often grouped together and assigned the 
benchmark of just one of the analyzed parts. This is done because an individual MALS 
has hundreds, potentially thousands, of parts that it repairs. The process used to 
benchmark one individual repairable is time consuming and obligates valuable man¬ 
hours. To do this repeatedly for each repairable would severely impair the maintenance 
department. 
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When determining parts to be grouped together for the same TRR assignment, 
work centers will often choose a repairable that is generally representative of the typical 
repair process for parts in that work center. The benchmark for the individual part is then 
applied as the benchmark for a group of parts. This group of parts could be all parts a 
work center repairs or groups of similar parts a work center repairs. 

Grouping parts together and assigning them all one single TRR for maintenance is 
inherently problematic, however. To illustrate this, consider a notional but reasonably 
realistic example of a work center that repairs composite main and tail rotor blades for 
two different types of helicopters. These four different types of blades have different part 
numbers and each blade has a multitude of types of repairs that may be required. Each 
blade can only be assigned one TRR and this TRR does not change with different types 
of repairs. Given the nature of composite repair and parts availability for Marine Corps 
rotor blades, some repairs may take days while some take weeks. 

Prioritization of the maintenance workload is based on the physical buffer status 
of an item and the time buffer of an item, its TRR. If the physical buffer status on two 
items repaired by the same shop is equal, the one with the shortest TRR will take priority 
if shortest processing time is the shop’s dispatch rule. If these are high priority parts, then 
the estimated TRR is also communicated to the AMO, ASO, and requisitioning squadron 
for other planning purposes. If the repair is estimated to take so long that it impairs 
readiness at the squadron, the AMO and ASO may seek an alternate solution to obtaining 
an RFI part. In the example of repair times for rotor blade repair, cannibalization of 
blades or lateral support requests are based upon the maintenance TRR of the item. 

Using four different rotor blades as an example, notional minimum and maximum 

total delay time of each rotor blade are demonstrated in Table 5. The 90th percentile of 

each blade’s repair time distribution is its TRR. Specifically, this means that 90% of the 

time, this blade will be repaired in that number of days or fewer. Using an average of 

repair distributions is a common way to assign multiple parts a single TRR. In this 

example, 21.58 days is the average of all the blades’ repair distributions. The cumulative 

density function (CDF) indicates the total probability of repairing that blade in a time 

frame less than or equal to 21.58 days . The 90th percentile and averaged TRR of all parts 
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was simulated using Crystal Ball. Beta distributions of repair times for eaeh blade were 
assumed and 100,000 trials were run per simulation. 


Table 5. Repair times for Notional Parts, TRR at 90th Pereentile, and CDF of Eaeh 

Part if Assigned Averaged 90th Pereentile. 



Min Repair 
(Days) 

Max Repair 
(Days) 

90% 

(TRR) 

CDF at 
21.58 days 

Blade A 

3 

45 

31.58 

60.787 

Blade B 

2 

25 

17.63 

99.045 

Blade C 

1 

15 

10.5 

100.000 

Blade D 

5 

65 

45.7 

31.341 


A foreeast was modeled for eaeh variable individually and a fifth foreeast was 
modeled of the average of the four variables. The 90th pereentile of eaeh blade’s 
individual foreeast is demonstrated in Table 5. The 90th pereentile of the averaged 
foreeast was 21.58 days. If 21.58 days is applied to each blade as the TRR, managing the 
maintenance workload based on TRR becomes problematic. Two of the blades will 
consistently exceed its TRR and two will consistently finish before its TRR. One can see 
in Figure 17 that Blade C will always meet its TRR while Blade D will only meet its TRR 
31% of the time. If any one of the TRRs is applied to all blades, rather than averaging the 
forecasts, the results are no more helpful. 
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Figure 17. Forecast Blade Repair Time Distributions and Average. 
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There is no hard requirement for work centers to ensure the accuracy of their 
benchmarks. According to the NAMP, the MALS ought to utilize the AIRSpeed team to 
“conduct value stream mapping and analysis to benchmark existing processes and 
illustrate improvement opportunities” (Department of the Navy, 2012, p. 3-55). This is 
the only requirement for developing benchmarks for repair processes. Ensuring these 
benchmarks are updated and reasonable is only implied at best and left to the 
responsibility of the AIRSpeed team. 

If a MALS optimizes the buffer status metric produced by the BMT, the resulting 
goal is to reduce buffer status red parts on the supply shelf. In order to reduce buffer 
status red parts, the maintenance department will prioritize all MAFs for parts that are 
red-coded for supply. Red-coded MAFs will always be worked first, followed by yellow 
and then green. Any MAFs that have exceeded their TRR are coded black for 
maintenance will be worked into the red-yellow-green rotation of MAF prioritization. 

To effectively manage the workload in this way. Design TRR estimates need to be 
reliable and specific to each part. Design TRR numbers need to be established for each 
part individually and updated as processes or capabilities change. Otherwise certain parts 
may always exceed their TRR and be consistently coded black. A maintenance manager 
will typically follow a specific dispatch rule, whether explicitly stated or not. If reducing 
red physical buffer status MAFs is the priority of the MALS, a shortest processing time 
dispatch rule may be adhered to. If multiple parts are assigned a non-specific Design 
TRR, a part may be inordinately prioritized. 

As holes get filled on the supply officer’s shelf (i.e., buffer status red parts are 
repaired and replenished), the squadron has an increased number of requisitions filled 
immediately, rather than be coded as not-in-stock. This means squadrons are both 
receiving parts and resuming work on aircraft more quickly. The MALS’ focus on filling 
holes in the supply shelf positively affects its fill rate. 

2. Time to Reliably Replenish—TRR 

The Time to Reliably Replenish metric is directly related to the previous 
discussion regarding the Buffer Management Tool. It can also be used as standalone 
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metric, however. According to the NAMP, TRR is generally defined as “the time, in 
hours or days, it takes once a part is pulled from the supply shelf until it is back on the 
supply shelf ready for issue” (Department of the Navy, 2012, p. A-79). Specifically, 
however, it means different things between Maintenance and Supply. 

For maintenance, it is the time to reliably repair a part that it receives. For supply, 
it is the time to replenish a part in inventory once it has been issued to a flight line 
squadron. Broadly speaking, TRR is the time it takes to order a part, receive it, and place 
it on the shelf as ready for issue. To compute the TRR, the times to fill historical 
requisitions of a particular item are measured. The 90th percentile of those times is then 
applied as the TRR for that part (Seagren, 2013). 

TRR is an important measurement in Marine Corps Aviation. MALSP MOD is 
structured around the ability to maintain consistent TRR measurements between various 
nodes within the logistic support framework. MALSP MOD is designed to reduce the 
supply footprint in forward deployed locations. According to MARADMIN 175/16, 

MALSP MOD is an improved expeditionary AVLOG concept that will 
provide more responsive, scalable, and flexible solutions through 
properly-sized spares packages while achieving near real time global 
visibility through an enhanced information technology (IT) and nodal 
logistics laydown (NLL) supply chain concept. (Headquarters Marine 
Corps, 2016, p.l) 

In order to ensure a certain service level for flight line squadrons with a nodal, 
demand-pull logistics network, the TRR between nodes in the supply chain need to be 
relatively reliable and consistent. If the TRR between nodes contains great variance, 
inventory buffer levels have to be increased. Maintaining a large inventory at 
the different nodes is one of the very situations that a nodal logistics network is meant 
to avoid. 

Optimizing TRR in this context can refer to either reducing the time it takes for 
maintenance to replenish the supply shelf or the time it takes for the supply chain (often 
the Defense Logistics Agency) to replenish the supply shelf. This metric does not 
measure the time to fill squadron requisitions. Optimizing this metric can be approached 
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in two different ways. First, it can mean reducing the TRR to as low of a number as 
possible. Second, it can mean making TRR as stable as possible. 

The first is only helpful as long as variation is minimal. The second approach is 
more helpful, even if the TRR is larger. This is because the supply officer can maintain 
higher service levels with a reliable TRR through buffer size management than if the 
TRR is unreliable and widely varying. High variance in TRR requires larger inventory 
buffers to maintain satisfactory service levels during replenishment times, which is not 
always possible due to parts availability and budgetary constraints. 

To demonstrate this, suppose the ASO desires a service level of 90%, demand for 
part X is constant with one per day, and lead time for part X (i.e., TRR) is not constant. A 
service level of 90% has a z value of 1.28. To determine the proper amount of safety 
stock (SS), the z value of the service level (z) is multiplied by the standard deviation of 
demand during lead time (cri) (Chase, Jacobs, & Aquilano, 2004, p. 557). Because 
demand is constant in this example, the standard deviation of demand during lead time 
(DDLT) will only increase as the standard deviation of lead time (i.e., variation) 
increases. 

Table 6 summarily presents two scenarios that portray the two ways in which 
TRR can be optimized, reducing lead time and reducing variation in lead time. In 
Scenario A, AVLOG reduces mean TRR without also reducing the variability of TRR. In 
this example, TRR might be reduced to four days and the standard deviation of that 
replenishment time may also be four days. To maintain a service level of 90%, a safety 
stock of approximately five items needs to be held on the supply shelf. 

Suppose in Scenario B, AVLOG chooses to reduce variability of TRR, rather than 
the mean TRR. In doing so, the resulting safety stock requirement is quite different. If 
TRR is eight days but variability of the replenishment time is two days, approximately 
two and a half items are needed as safety stock on the supply shelf to maintain a 90% 
service level. The TRR might be twice as much in this scenario but variability of that lead 
time is half, resulting in half the safety stock required. 
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Table 6. Safety Stock Requirements with Different TRR Variability 



Scenario A 

Scenario B 

Demand During Lead Time (1/day) 

4 

8 

Standard Deviation of Lead Time (oL) 

4 

2 

Service level desired (z) 

90.00% 

90.00% 

Safety Stock (SS = z * oL) 

5.13 

2.56 


Optimizing TRR through reduced variation can therefore be more beneficial than 
reducing mean TRR but allowing increased variation in replenishment times. 
Understanding the implications of variance in TRR is therefore essential for proper buffer 
size management. If buffer sizes are managed appropriately, TRR itself does not 
necessarily have any positive or negative impact on squadrons. The squadron is only 
negatively affected if they order a part that is not in stock. Whether a part’s TRR is 
60 days or 10 days, the squadron is only affected if the ASO’s buffer is not sufficient to 
cover that transportation time. 

3. Beyond Capability of Maintenance 

The maintenance department occasionally encounters situations where they have 
a part that they are unable to repair. When this situation arises, the MAP receives an 
action code of Beyond Capability of Maintenance (BCM). There are various reasons a 
MAPS may not be able to perform maintenance on a part. Examples include lack of 
parts, lack of repair equipment, or beyond authorized repair depth. The BCM codes, 
therefore, have accompanying identifiers to describe why the MAPS is unable to perform 
the repair. These codes are identified in Table 7. 


Table 7. BCM Codes 


BCMl 

Repair Not 
Authorized 

BCM 4 

Lack of Parts 

BCM 7 

Beyond Authorized 
Repair Depth 

BCM 2 

Lack of Equipment, 
Tools, or Facilities 

BCM 5 

Fails Check and Test 

BCM 8 

Administrative 

BCM 3 

Lack of Technical 
Skills 

BCM 6 

Lack of Technical 
Data 

BCM 9 

Condemned 
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Parts being coded BCM are of concern due to the general higher cost of 
conducting repair at a higher level of repair, including the Depot level or manufacturer. It 
is generally cheaper to repair a part at the lowest level possible, with the flight line 
squadron being the lowest, followed by the intermediate level MALS. The maintenance 
department and supply department are both interested in BCM rates, albeit for different 
reasons. The supply department is primarily interested in BCM 4 codes because that is 
the code for which they have the most influence. The maintenance department, however, 
is concerned with all BCM codes. 

High BCM rates may indicate that either the maintenance or supply department is 
unable to perform its job adequately. A maintenance department reporting a BCM 3 does 
not reflect positively on the maintainers within that MALS. If a MALS must BCM a 
repairable, the BCM 7 code is the one preferred. This is because it indicates that they 
have sufficiently done all they are authorized to do and are required to submit the part for 
a higher level of repair. 

The BCM metric incentivizes cost-reduction through in-house repair of parts. 
Focusing on keeping low BCM rates, however, may diminish MALS support to flight 
line squadrons. There are often long periods of troubleshooting and repair before an item 
receives a BCM code (with the exception of BCM 1 and most BCM 7 codes). If the goal 
is to minimize BCM codes, then those periods of troubleshooting and repair will be 
further extended as maintainers continue to attempt successful repair. These longer repair 
times could potentially result in longer wait times for the flight line squadron who 
requires the part. 

Optimizing BCM rates means to reduce the number of BCMs that occur, 
primarily BCM 4 and BCM 7. BCM 7 rates can be reduced through BCM interdiction, 
when a MALS receives authorization to conduct maintenance themselves or a local 
artisan that is not typical of the I-Level. BCM 4 rates can be reduced through parts 
acquisition and inventory management. 

BCM rates only positively affect the flight line squadron if it results in a 
ready-for-issue part being issued to the flight line squadron more quickly than had the 
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part been sent off for repair and a ready-for-issue part been requisitioned. Low or high 
BCM rates themselves do not neeessarily affeet fill rates for flight line squadrons. As 
long as a MALS has spare parts on the supply shelf for the squadron, BCM rates are 
irrelevant (in terms of flight line squadron support). 

4. Supply Effectiveness or Perfect Order Fulfillment 

Supply effectiveness, or fill rate, is primarily a supply metric, measuring what 
percentage of demands are filled immediately upon requisition. Items that are not in stock 
or not carried in the supply system detract from the total percentage. If a requisitioned 
item is in stock, then that completed requisition will contribute to the total percentage. 
This metric is typically described as net supply effectiveness and gross supply 
effectiveness. Net supply effectiveness considers only items that are carried while gross 
supply effectiveness considers all requisitions. Balestreri and McDoniel (2002) define 
perfect order fulfillment in much the same way as supply effectiveness, though without 
net and gross distinctions. 

This metric is beneficial for several reasons. First, it is easily calculated. Second, 
high fill rates have corresponding tangible effects. Third, it most easily discussed and 
communicated. There can be, however, negative aspects of this metric. Primarily, this 
metric can treat all requisitions as equal, requisitions that render an aircraft non-mission 
capable (coded AKO) or partial-mission capable (coded AK7). In order to capture this 
disparity, the ASO must filter for requisitions that are coded AKO and AK7. Further 
filtering between consumables and repairables also improves the helpfulness of this 
metric, as this delineates between who may be responsible for filling a hole on a shelf 
(i.e., maintenance or supply). 

Another negative aspect of this metric is the absence of any time dimension 
consideration. An AKO requisition that is not immediately filled will degrade the supply 
effectiveness measure. If that requisition takes 10 or 40 days to fill, however, is not taken 
into account. These two scenarios have very different second order effects on a squadron. 
Dispatch rules that properly prioritize these outstanding requisitions by examining the 
time dimension of these requisitions will best impact the squadron. 
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Optimizing the supply effectiveness measure means ensuring that when a part is 
ordered, it is in stock. This often equates to minimizing holes on the supply shelf. Priority 
should be given to holes on the shelf whose respective parts are ordered the most 
frequently, however. The original MALSP model focused on optimizing fill rate, which 
resulted in greater range and depth on the supply shelf. The end result of such focus 
resulted in large inventories forward deployed. 

Supply effectiveness connects the MALS very directly to the squadron and high 
supply effectiveness rates enable the squadron to best work towards improving flight line 
readiness. The effects of improved supply effectiveness are easily seen. It results in flight 
line squadrons receiving parts more often, that is, having less not-in-stock coded 
requisitions. 

Focusing on supply effectiveness as a single metric, however, places the burden 
primarily on supply. Maintenance is not able to directly influence this fill rate without 
receiving direction from Supply. While maintenance does replenish the supply shelf, 
supply is still the first point of contact in answering for low supply effectiveness 
performance. Supply effectiveness is therefore a reasonable measure for the ASO to 
examine but less useful for the MALS as an entity. 

5. IMA Effectiveness 

An Expeditious Repair (EXREP) is when an intermediate maintenance activity 
(IMA) receives a non-REI repairable part from a squadron and, because that part is 
rendering an aircraft NMCS or PMCS, needs to be repaired expeditiously. An EXREP is 
defined by the NAMP in this way: 

The processing for repair of NIS or NC components (repairable or 
consumable). These components must be in support of, or related to, an 
NMCS or PMCS, situation. This processing is accomplished by the 
immediate removal of the component from the aircraft, expedited delivery 
and induction for repair, and the earliest return to REI status for supply 
issue under the standard material issue priority system. (Department of the 
Navy, 2012, p. A-25) 

In a similar way that net or gross supply effectiveness is used as a performance 

metric for the supply department, the maintenance department may use EXREPs as a 
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basis for measuring performance. This metric is called “IMA Effectiveness” and is 
computed by dividing the number of EXREPs the IMA receives and how many of those 
are repaired and reissued as REI. An EXREP occurs because there is not an asset on the 
Supply Officer’s shelf that can be immediately issued as REI or the asset is not carried. 

Optimizing this metric in the maintenance department will lead to the immediate 
needs of the squadron being met as long as the parts are repairable at the IMA level. 
Because the parts are immediately necessary to help resolve and NMCS or PMCS 
discrepancy, quick turnaround on EXREPs can more directly be linked to improved 
aircraft readiness than working on MAPs that fill yellow physical buffer status items on 
the supply shelf. 

EXREPs, however, are indicative of holes on the supply shelf. A perfectly 
optimized supply chain may render EXREPs nonexistent. While maintenance will do 
well to ensure EXREPs are repaired and reissued quickly, the MAES could potentially 
avoid EXREPs if resources were effectively allocated towards physical buffer status red 
reductions, forecasting, and inventory management. 

To achieve a high IMA Effectiveness ratio, a MAES must repair and reissue 
EXREPs within the same month they are issued, or reduce the number of EXREP 
occurrences. The former is accomplished through expedited repair processes by 
maintenance, the latter through physical buffer status red reduction by maintenance and 
supply jointly. Reducing EXREP occurrences more positively affects the squadron than 
expedited repairs because this metric is measured in terms of EXREPs per month, not 
time to complete an EXREP. If an IMA receives an EXREP on the first day of the month, 
the MAES will receive the same measure for IMA effectiveness whether they take 
28 days or two days to reissue that item. 

6. Supply Response Time 

Supply Response Time (SRT) is a fairly direct metric and focuses specifically on 
the time between when a part that is on-station is requisitioned and when the squadron 
receives the part. High priority items, for example, have a target supply response time for 
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part delivery of one hour. This measurement is generally provided as an average of all 
delivery times or as a percentile. 

Supply, however, does not have total control of the factors that affect SRT. 
Misaligned schedules of the MALS and the flight line squadron can render it impossible 
to deliver parts at certain times. A flight line squadron’s operating schedule is more 
dynamic than a MALS operating schedule. For example, if a flight line squadron is 
operating on a limited schedule due to some personnel being gone for training 
detachments, there may not be personnel at the squadron to accept any delivered parts. 
This negatively impacts supply’s SRT. 

Optimizing SRT in its current construct (the positive measure of how many 
requisitions are delivered in the target time) may not have any remarkable effects on 
flight line squadrons. SRT is a positive measurement and thus is indicative only of 
deliveries made successfully within the target time. In this manner, it is a function of 
parts already on the supply shelf. 

An SRT of 90% parts delivered within the target time is misleading regarding the 
level of service provided to the squadron. This is because the 10% of parts that were not 
delivered within the target time are unaccounted for in this metric. Furthermore, a portion 
of that 10% may be just outside of the target SRT. Delivering a part in four hours, when 
its target time was one hour, will not have any significant effect on aircraft readiness. 
Measuring only the on-station time of a requisition is therefore only marginally 
beneficial. 

7. Supply Chain Response Time 

The Supply Chain Response Time (SCRT) metric, as suggested by Balestreri and 
McDoniel (2002), seeks to resolve SRT’s shortcomings in regard to accounting only for 
on-station time of a requisition. They define on-station supply response time as logistics 
response time and and the off-station time of a requisition as customer wait time. These 
two measurements are combined in the SCRT. This is helpful because supply response 
time examines only a portion of the logistics system’s activity. 
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If SCRT is made a function of parts that were not delivered in their target delivery 
time frame, then the metric becomes much more useful and beneficial for both the MALS 
and the flight line squadron. By capturing this time as a negative measurement (rather 
than the positive measurement of how many items were delivered in the target time 
frame), the potential impact of delayed delivery can be better quantified and 
communicated. To do so, the MALS would measure only the parts that missed their 
target delivery time. 

Categorizing those off-station requisitions according to their physical buffer 
status would further improve this metric. SRT construed in this way would be a negative 
measurement. As a negative measurement, it is much more indicative of the quality of 
support a MALS provides the squadron. 

SCRT better affects flight line squadrons if taken as a negative measurement 
rather than a positive measurement. Focusing improvement efforts on reducing excessive 
delivery time results in the flight line receiving critical parts more quickly. Information of 
this sort will help the ASO allocate his resources more effectively to improve support to 
the flight line. 


8. MALS Effectiveness 

MALS Effectiveness is a type of performance measurement system used by 
several MALS seeking to provide a holistic view of maintenance and supply functions 
in one metric. The MALS Effectiveness metric is calculated by averaging the 
supply effectiveness metric (i.e., filled requisitions) and the IMA effectiveness metric 
(i.e., repaired EXREPs). If a MAES supply effectiveness metric is 80% and the IMA 
effectiveness metric is 60%, then MAES effectiveness is 70%. While supply 
effectiveness captures a large amount of the output from supply, EXREPs are only a 
small part of the output from maintenance. 

Eor example, if a MAES has 800 requisitions and supply fills 700 of them, its 
supply effectiveness is 87.5%. If it has 30 EXREPs and reissues 15 and BCM codes the 
other 15, its IMA effectiveness is 50%. The MAES effectiveness of this MAES is 
therefore 68.75%. If supply achieved a measurement of 100% (i.e., filling the remaining 
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100 requisitions), MALS effectiveness would only increase to 75%. If supply’s 
performance remained the same, maintenance would have to achieve a measurement of 
62.5% (i.e., repairing only four more EXREPs) to achieve 75%. 

Because of its disproportionate nature, focus on improvement to this metric may 
not greatly increase support to the squadron. To increase MAES Effectiveness in the 
previous example, maintenance may only need to REI between five and ten EXREPs. 
Eive to 10 parts out of 830 requisitions and EXREPs will benefit the squadron but not 
greatly. 

E. MEASUREMENTS AND BEHAVIOR 

Measurement systems will affect the behavior of those being measured. It has 
been discovered, however, that some measurement systems will have unintended 
consequences for overall performance. A result may be “that the types of behavior 
rewarded are those which the rewarder is trying to discourage, while the behavior is not 
being rewarded at all” (Kerr, 1995, p. 7). Another disparity between measurements and 
behavior can occur when a positive behavior is rewarded but “provides no actual benefit 
for the system” (Doerr & Gue, 2012, p. 726). This can occur when a dispatch rule or 
policy is internally focused and does not directly contribute to system output. 

This section focuses primarily on examining current primary metrics used by the 
MAES and how they can influence the behavior of Marines in a MAES. Each metric has 
examined in detail in this section also in order to fully understand the metrics and how 
they are calculated. Eurther emphasis is placed on the behavior that each metric 
encourages within the MAES. 

1. Buffer Management Tool—Buffer Status 

The focus of the BMT system is to rightly prioritize maintenance in order to 
ensure the supply shelf is adequately stocked. By establishing a reliable benchmark for 
repair time of part, more reliable predictions can be made regarding repair completion. 
This is helpful for both the ASO and the squadron that requires that part. The ASO is 
better able to manage his inventory and buffer size for that part in order to ensure 
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continued availability of that part for the squadron. If parts take longer to repair than the 
benchmark then the ASO may not have the part when the squadron needs it. If the repair 
is repeatedly faster than the benchmark, then the ASO may have unnecessary extra parts 
on the shelf. 

Proper use of the BMT will incentivize Marines to prioritize MAFs that, once 
completed, will fill empty spots on the ASO’s supply shelf. The supply shelf becomes the 
customer that the maintenance department serves. This does not always directly correlate 
to the greatest needs of flight line squadrons, however. 

Not all empty shelf spots have equal importance. The frequency and quantity at 
which an item is ordered dictates the level of importance it demands. An item historically 
requisitioned once a year will have a supply buffer status of red until it is replenished. An 
item requisitioned once a day will have the same buffer status. The buffer status color 
code itself does not indicate which item ought to take priority. 

Optimizing BMT can have positive and negative effects on Marines’ behavior. 
Positively, it theoretically encourages Marines to expedite repair or processing of the 
most needed parts—those that are buffer status red. Negatively, increasing the range of 
the supply shelf (i.e., adding parts to carry in inventory) may be discouraged. By 
increasing the number of parts that carried, there are more opportunities to have 
occurrences of buffer status red. 

Design TRR numbers are easily manipulated and if a part repeatedly exceeds its 
Design TRR, the TRR can simply be increased with much less effort than attempting to 
improve the repair process. If BMT is optimized. Design TRR will have to be accurate. 
Ensuring these numbers are accurate is manpower intensive (under the current construct). 
Allocating resources to ensure Design TRR numbers are current and accurate incurs great 
opportunity cost, as every man-hour spent on such a task is one less man-hour spent on 
repairing parts. 
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2. Time to Reliably Replenish—TRR 

Focusing on TRR as a primary metric for a MALS may result in increased 
inventory levels because to account for the variation of supplier or maintainer times 
(direct factors of supply’s TRR), a larger buffer of spares is required. With increased 
inventory levels come increased costs. This was the model of MALSP I but is contra to 
the underlying principle of MALSP MOD. 

However, it does increase the level of support provided to supported squadrons. 
Reduced TRR and increased inventory results in squadrons having what they need more 
quickly. In a fleet of aging aircraft, however, increasing inventory of certain dynamic 
components can be expensive at best and infeasible at worst. 

In order to optimize TRR, the processes for forecasting shortages must be 
improved. Currently, there is no framework for forecasting what may reach buffer status 
red without timely replenishment. An algorithm that factors historical demand, current 
buffer status, and estimated time to delivery can provide a rough indication of whether a 
part will arrive before it is requisitioned by a squadron. Any parts indicated by such a 
calculation could then be opportunities for the ASO to begin expediting before they 
become needed for down aircraft. 

3. Beyond Capability of Maintenance 

Optimizing BCM generally means making every effort to repair parts in-house, 
also called BCM interdiction. A likely result of this is elongated repair times, as a MALS 
spends more time troubleshooting a problem or seeking authorization to conduct certain 
repair procedures normally reserved for higher levels of maintenance. If these parts are 
needed by the squadron, then delivery may be delayed. 

A reasonable MALS will not conduct BCM interdiction on a part that is needed 
immediately. However, if BCM interdiction begins first on a low priority part and then 
the part becomes high priority, delivery of an RFI part may be delayed due to previously 
started BCM interdiction. In this scenario, BCM interdiction can have negative 
consequences. 
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BCM interdiction can also be problematic when it requires resources that may 
otherwise be better spent on other maintenance. Determined Marines may spend an 
inordinate amount of time trying to repair a part when it could be more efficiently 
repaired elsewhere. While BCM-ing a part may be more expensive, reduced aircraft 
readiness may arguably be more expensive in terms of opportunity cost. 

4. Supply Effectiveness or Perfect Order Fulfillment 

Optimizing supply effectiveness encourages Marines to increase inventory buffers 
in order to ensure parts are available when requisitioned. This leads to increased 
inventory and holding costs. However, because supply effectiveness is generally reported 
in terms of Net or Gross terms, increasing the range of the supply shelf may be 
discouraged. 

Supply effectiveness is a measurement that attempts to capture the output of a 
MALS to its customer. However, similar to SRT, it does not adequately capture the 
negative impact of the requisitions that go off-station due to being not-in-stock or not 
carried. The vast majority of conversations between the Maintenance Officer, Supply 
Officer, and squadron Maintenance/Material Control Officers revolve around those off- 
station parts. 

5. IMA Effectiveness 

The IMA Effectiveness metric positively increases by repairing and reissuing 
EXREP parts to the squadron in the same month that they are received. A MAES will 
receive a 100% IMA Effectiveness if they receive and reissue one-of-one EXREPs or 30- 
of-30. Reducing EXREPs, therefore, does not decrease IMA Effectiveness. 

If this metric is used as a focal point for performance measurement then all of a 
work center’s resources will be dedicated to that EXREP when it is received. If the 
EXREP cannot be repaired by the work center, then that part receives a BCM code and is 
sent to a higher level of maintenance. A MAES generally makes every effort to quickly 
return EXREPs to the squadron or quickly BCM the item in order to requisition an REI 
asset. 
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When an EXREP is received, a work center’s attention is turned away from its 
current task to repair the inducted EXREP item. This pulls manpower away from 
maintenance on other items, potentially critical buffer status red items. This is the only 
case when allocated resources to an EXREP item may prove problematic. Even so, 
EXREPs are generally a very small portion of an IMA’s total MAE count and therefore 
do not drastically impair progress on current maintenance being conducted. Because 
EXREPs are such a small portion of an IMA’s workload, using them as a metric for 
effectiveness falls short of capturing the IMA’s activity as a whole. Supply effectiveness 
is a much more holistic measure of the supply department than IMA Effectiveness is of 
the maintenance department. 

6. Supply Response Time 

Eocusing on SRT may lead to expeditious delivery of priority parts that are in 
stock. As requisitions are placed, high priority parts (determined by their priority code) 
will be delivered first because they have the shortest time requirement. Eower priority 
parts have longer target times to be delivered. 

SRT is normally calculated as a percentage of requisitions that were filled within 
the target timeframe. In this regard, it is a positive measure communicating the successes 
of the supply department. SRT construed in this manner can negatively affect supply 
personnel behavior, however. 

A MAES supply department may generally have an SRT success rate of 90%. 
This means that 90% of all requisitions were delivered within the target timeframe. 
Eraming the metric in this positive manner does not communicate the negative affect the 
remaining 10% may have on the flight line squadrons. A positive metric of 90% may 
create a culture of complacency within Marines as it diverts the focus from improvement 
opportunities. 

If the metric is communicated in a negative sense, it will focus on the remaining 
10% that did not meet the target time frame. Eurthermore, if the metric is communicated 
not as a ratio but as a time, then the information is much more meaningful. Examining a 
certain percentile of delivery times of the 10% that did not meet the target time frame not 
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only highlights improvement opportunities but better eommunicates how the flight line 
squadrons are being affected. 

7. Supply Chain Response Time 

SCRT measures the total time on- and off-station of a requisition. If the 
measurement is reported negatively (i.e., only measuring those requisitions that missed 
their target delivery time), then the MALS can gain a better understanding of how the 
logistics network is impacting the flight line squadron. A negative measurement better 
communicates to the logistics system how it may be hindering aircraft readiness. It also 
more clearly identifies where process improvements can be made. 

In the same way that SRT reported positively may create a sense of complacency 
among Marines, SCRT may do the same. If a SCRT success rate of 90% is reported, then 
the MALS is delivering 90% of requisitions in the target time. However, the remaining 
10% are the requisitions that are immediately affecting the flight line. Separating the 
remaining 10% into high and low priority documents can better focus MALS’ efforts on 
the critical few parts that are negatively affecting the flight line. 

Because SRT only measures on-station time, and SCRT combines both on-station 
and off-station time, SCRT is much more helpful as a measurement. While it is true that a 
MALS does not fully own any of the off-station supply chain processes, it does have a 
great deal of influence on them. Expediting requisitions, negotiations with item 
managers, and trade-offs with other MALS are all ways that a MALS can reduce off- 
station time of a requisition. 

8. MALS Effectiveness 

Giving equal weights to supply effectiveness and IMA effectiveness, the two 
metrics that are averaged to determine MALS effectiveness, is disproportionate. A much 
greater amount of work must be done by the supply department than the maintenance 
department to increase the metric. Only a small portion of the IMA’s workload is 
captured by this metric. Because of the disproportionate nature of this metric, optimizing 
this metric may result in demoralizing supply department Marines due to the 
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disproportionate amount of work they must exert to increase their effectiveness in 
comparison with the maintenance department. 

In the maintenance department, Marines’ behavior may not change very 
drastically. EXREPs are a small portion of their regular workload and only receive 
attention when they are received. Once received, they are prioritized above the regular 
workload and every effort is made to repair the item as quickly as possible. If it cannot be 
repaired in-house, it is BCMed and is sent to a higher level. Once either of these events 
takes place, the Marines return to their regular workload. Because EXREPs are not part 
of their regularly managed workload, focusing on MAES Effectiveness will not 
incentivize greater production on a day-to-day basis for work centers. 

F. CRITERIA APPLIED 

To evaluate and compare the discussed metrics, Caplice and Sheffi’s (1994) 
evaluation criteria are used and adapted for relevancy to the MAES. Table 8 summarizes 
the criteria and provides examples of the types of questions asked regarding each metric. 
Each metric is then scored according to these criteria. A score of “5” indicates that the 
criterion describes that metric very well. A score of “1” indicates that the criterion poorly 
describes that metric. The scores are then averaged for a total score of that metric. 


Table 8. Evaluation Criteria. Adapted from Caplice and Sheffi (1994). 


Validity 

Does it capture the events and activities being measured and controls 
for external factors? 

Robustness 

Is it interpreted similarly by users? It is comparable between MAES? 

Is it repeatable? 

Usefulness 

It is readily understandable by decision maker? Does it provide a 
guide for action to be taken? 

Integration 

Does it include all relevant aspects of the process? Does it promote 
coordination across functions? 

Economy 

Does the benefit of using the metric outweigh the costs of data 
collection, analysis, reporting? 

Compatibility 

It is compatible with existing information, material, and systems in 
the organization? 

Level of Detail 

Does it provide a sufficient degree of granularity or aggregation for 
the user? 

Behavioral Soundness 

Does it minimize incentives for counter-productive acts or game¬ 
playing? Is it presented in useful form? 
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1 . 


Buffer Management Tool 


The BMT captures a great deal of data regarding maintenance and supply 
functions. The Physical Buffer Status Red metric is one of the most often used metrics 
produced by this system. The data is easily collected and the metric is easily 
communicated. It quickly focuses maintenance and supply leaders on critical needs on 
the supply shelf. 

However, due to the variety of squadrons supported by different MALS, the 
number of physical buffer status red items does not communicate what level of support is 
provided to squadrons in comparable terms. Physical buffer status red also does not 
communicate information regarding what parts are currently needed on the flight line. An 
item does not have to be requisitioned in order to have a red physical buffer status. Table 
9 summarizes the evaluation of this metric according to the proposed metric criteria. 


Table 9. Metric Criteria and BMT 


Metric Criteria 

Buffer 
Status Red 
(BMT) 

Validity 

5 

Robustness 

3 

Usefulness 

4 

Integration 

4 

Economy 

4 

Compatibility 

5 

Level of Detail 

5 

Behavioral Soundness 

4 

Sum 

34 


2. Time to Reliably Replenish 

The MALSP MOD nodal logistics laydown framework is largely built upon the 
TRR concept and thus this metric is a commonly understood idea across AVLOG 
organizations. TRR can be defined in both maintenance and supply terms. As a 
maintenance concept, it is founded on the Design TRR of parts. The Design TRR is easy 
to manipulate, difficult to maintain accuracy, and arbitrary if applied to multiple parts 
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based on a single part’s analyzed TRR. As a supply measurement, it is founded on the 
historical replenishment times of parts through the supply chain, and is therefore more 
reliable and useful. 

TRR does not measure the output of a MALS and does not reflect the quality of 
support a MALS provides to squadrons. While it is a helpful metric for a sub-function in 
AVLOG, it is less helpful when measuring MALS performance. Table 10 summarizes the 
evaluation of this metric according to the proposed metric criteria. 


Table 10. Metric Criteria and TRR 


Metric Criteria 

TRR 

Validity 

3 

Robustness 

3 

Usefulness 

3 

Integration 

3 

Economy 

2 

Compatibility 

5 

Level of Detail 

3 

Behavioral Soundness 

2 

Sum 

24 


3. Beyond Capability of Maintenance 

BCM is a metric that is used in every MALS but is not directly comparable 
between different MALS. This is due to the different repair capabilities and support 
requirements at each MALS. For example, new aircraft may have more parts that have to 
be automatically BCM coded than legacy aircraft parts due to proprietary information of 
manufacturers. It is easily calculated and communicated, however. 

BCM accurately captures a portion of MALS activity but it does not capture 
MALS output. Rather, it measures when maintenance is not able to repair a part in-house. 
Because a BCM part does not directly affect flight line squadron support, it is less helpful 
in measuring MALS performance. Table 11 summarizes the evaluation of this metric 
according to the proposed metric criteria. 
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Table 11. Metric Criteria and BCM 


Metric Criteria 

BCM 

Validity 

4 

Robustness 

3 

Usefulness 

4 

Integration 

2 

Economy 

3 

Compatibility 

5 

Level of Detail 

4 

Behavioral Soundness 

3 

Sum 

28 


4. Supply Effectiveness or Fill Rate 

Supply Effectiveness is a commonly used and well understood metric. It is 
computed easily and directly measures the output of a MAES. It is comparable between 
MAES but is most helpful if it is compared between T/M/S at each MAES. This will 
identify any potential systematic support issues in a T/M/S that are AVEOG wide issues. 

Presenting this as a positive metric (i.e., successes rather than failures) may 
discourage process improvement as this metric is generally quite high. Presented 
negatively with emphasis on NMCS and PMCS parts, however, focuses leaders’ attention 
on the critical few items that are negatively affecting the flight line. The evaluation of this 
metric according to the proposed metric criteria is summarized in Table 12. 


Table 12. Metric Criteria and Supply Effectiveness 


Metric Criteria 

Supply Effectiveness 

Validity 

4 

Robustness 

4 

Usefulness 

4 

Integration 

4 

Economy 

4 

Compatibility 

5 

Level of Detail 

4 

Behavioral Soundness 

3 

Sum 

32 
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5. 


IMA Effectiveness 


IMA Effectiveness only captures a very small portion of IMA activity. This 
metric is not used by all MAES and is therefore not currently comparable between 
MAES. This metric is a factor of repair capabilities and support requirements and 
therefore is further limited in comparability between MAES. 

It is an easily calculated and understood metric, however, and may help the AMO 
and ASO identify process improvement areas. Because delivering an REI part to the 
squadron is the goal, it must be the focus of this metric. Therefore, the quickest route 
(i.e., BCM or repair) to delivery that REI part should be pursued. If this metric is 
optimized, it may encourage Marines to troubleshoot parts longer than it would take to 
order and receive a part once BCM coded. The evaluation of this metric according to the 
proposed metric criteria is shown in Table 13. 


Table 13. Metric Criteria and IMA Effectiveness 


Metric Criteria 

IMA 

Effectiveness 

Validity 

2 

Robustness 

3 

Usefulness 

3 

Integration 

2 

Economy 

3 

Compatibility 

4 

Level of Detail 

2 

Behavioral Soundness 

3 

Sum 

22 


6. Supply Response Time 

Supply Response Time measures delivery times against target times that are 
standardized for all MAES. These target times are only applicable to requisitions once 
they are on station. This limits the scope of this metric as it only captures a small portion 
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of the supply chain. It does not focus leaders’ attention on areas that can have significant 
impact on the quality of flight line support. 


Using this metric does not contribute greatly to improving flight line support if 
optimized. While this is the only portion of the chain that MALS completely owns, it is 
not the only portion with which they have influence, and thus this metric is too narrowly 
focused. The evaluation of this metric according to the proposed metric criteria is given 
in Table 14. 


Table 14. Metric Criteria and SRT 


Metric Criteria 

SRT 

Validity 

3 

Robustness 

3 

Usefulness 

3 

Integration 

2 

Economy 

2 

Compatibility 

5 

Level of Detail 

3 

Behavioral Soundness 

3 

Sum 

24 


7. Supply Chain Response Time 

Supply Chain Response Time is a more holistic measurement of the wait time a 
squadron faces from requisition to delivery. Focusing on NMCS and PMCS requisitions 
improves the usability and applicability of this metric. It can be easily compared between 
MALS though it should be subdivided for each T/M/S that the MALS supports. 

This metric is not currently used among the MALS, however. Also, it accounts for 
a great deal of time that is not controlled by the MALS. Using this metric may enable 
AVLOG leadership to better focus process improvement efforts on systematic issues. The 
evaluation of this metric according to the proposed metric criteria is shown in Table 15. 
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Table 15. Metric Criteria and SCRT 


Metric Criteria 

SCRT 

Validity 

4 

Robustness 

3 

Usefulness 

5 

Integration 

4 

Economy 

4 

Compatibility 

5 

Level of Detail 

4 

Behavioral Soundness 

4 

Sum 

33 


8. MALS Effectiveness 

MALS Effectiveness is not a standard metric used by all MALS. While it captures 
the majority of output from the supply department, it only narrowly captures the output 
from the maintenance department that it captures. A high priority on EXREPs is standard 
among MALS and thus this metric can be easily communicated and understood. 

Because this metric is an average of two other metrics (supply effectiveness and 
IMA effectiveness), the result is equal emphasis on each metric. This is misleading 
however and using this as a performance measurement may encourage counter¬ 
productive behavior among Marines. The evaluation of this metric according to the 
proposed metric criteria is shown in Table 15. 


Table 16. Metric Criteria and MALS Effectiveness 


Metric Criteria 

MALS Effectiveness 

Validity 

3 

Robustness 

2 

Usefulness 

2 

Integration 

2 

Economy 

2 

Compatibility 

4 

Level of Detail 

2 

Behavioral Soundness 

2 

Sum 

19 
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G. SINGLE, MULTIPLE, AND WEIGHTED METRICS 

A single logistic metric is limited in the information it can provide about an 
organization’s overall performance and is thus insufficient as a holistic performance 
measurement system. A MALS has many individual metrics available to measure its 
plethora of sub-functions. These combined sub-functions produce a corporate output. If a 
single metric must be used to measure the performance of the MALS, it should be the 
metric that captures this output most thoroughly and in terms that are indicative of the 
impact a MALS has on the flight line. 

No single metric currently used, however, perfectly captures the output of a 
MALS. Deliveries are the primary and majority output of a MALS but other smaller 
outputs exist. To capture all the outputs of a MALS, a performance measurement system 
consisting of multiple metrics must be used. Metrics that capture these outputs would be 
reported together as indicative of performance. A group of multiple metrics may be used, 
therefore, as a performance measurement system. Groups of metrics, however, can 
become confusing when being communicated. Additionally, if every output of the MALS 
is captured, the number of metrics may become cumbersome. 

Multiple metrics may then be combined into a single performance metric. This 
method has advantages and disadvantages. It captures multiple metrics into one and is 
therefore more easily communicated and briefed. However, if a simple average is taken, 
equal weight is being applied to all sub-metrics. This problem is seen in the MALS 
Effectiveness metric. 

Applying appropriate weights to each of the sub-metrics that are reflective of the 
amount of output or importance of the sub-function that is measured is one way to avoid 
this flaw of averages. The Navy Submarine Fleet Maintenance program (Submarine Fleet 
Maintenance Metric Working Group, 2013) approaches their performance measurement 
in this fashion. This method scales each metric according to the proper amount of 
influence or importance it has in the overall system. 

A system of appropriately weighed metrics is still not without problems, however. 
A high score on one metric may be negatively skewed by a very low score on another 
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metric. When this is the case, presenting only one combined metric will not present any 
indication of this problem. To control for this, each sub-metric may be assigned a goal 
and a lower control limit. If a sub-function is drastically underperforming (i.e., it is 
scoring below a sufficient level) then that can be identified through the use of a lower 
control limit. An overall combined performance measurement can still be presented and 
if there are any outliers that are underperforming, more detail may be provided. 

H. PERFORMANCE MEASUREMENT SYSTEMS 

The creation of a performance measurement system for a MALS ought to abide 
by the six criteria proposed by Caplice and Sheffi (1995), summarized in Table 17. Doing 
so will ensure the organization is able to make proper decisions to provide the best 
support to the customer. A performance measurement system views the organization in a 
holistic manner; a sum of parts rather than many separated processes. 


Table 17. Measurement System Criteria. Adapted from Caplice and Sheffi (1995) 

Measurement System Criteria 

Comprehensive 
Causally Oriented 
Vertically Integrated 
Horizontally Integrated 
Internally Comparable 
Useful 


A comprehensive performance measurement system for the MALS will include 
metrics that capture maintenance and supply department outputs as well as customer 
service. These are the primary stakeholders in a MALS and thus should be a focus of a 
performance measurement system. Not including stakeholder perspective in a 
measurement system will lead to actions taken that may negatively affect one of them. 

A causally oriented performance measurement system will measure activities that 
directly influence MALS performance. Performance, in this sense, is understood as 


65 




current and future performance (Caplice & Sheffi, 1995). Performance for a MALS is 
primarily a factor of RFI parts delivered to squadrons. 

A performance measurement system that is vertically integrated in a MALS will 
clearly communicate the goal and strategy of the MALS to department, division, and 
work center leaders. It will be reflective of the overall goal of the MALS. Marines will 
understand how their role contributes to improving performance because performance is 
clearly defined. 

A horizontally integrated performance measurement system in the MALS will 
include “pertinent activies, functions, and departments along the process” (p. 63). To do 
so, meaningful and causally oriented metrics from MALS’ major sub-functions need to 
be included. This results in greater integration and cooperation in a MALS. 

An internally comparable performance measurement system is one that can 
demonstrate how trade-offs can be made among processes in the MALS to increase 
performance. An example of these trade-offs may be how repairing physical buffer status 
red items and deliving parts may affect one another. If the measurement system is 
internally comparable, then the metrics affect and are affected by one another. 

Lastly, a performance measurement system ought to be useful to MALS 
leadership. It should be “readily understandable by the decision maker and provide a a 
guide for action to be taken” (p. 65). If it is too complex, the meaning of the metrics will 
be ignored. If the metrics do not seem meaningful, they will not be perceived as 
beneficial and therefore will not be used. 

I. MALS AND T/M/S 

No MALS is alike in terms of the T/M/S aircraft it supports. Because of these 
differences, metrics are not directly comparable MALS-to-MALS. The performance of a 
MALS that supports multiple H-1 squadrons cannot be directly compared to a MALS that 
supports multiple F-18 squadrons. This is due to the systematic differences of supporting 
certain aircraft. Systematic supply issues for a specific T/M/S can skew metrics unfairly. 
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In order to compare different MALS, the metrics should be MALS and T/M/S 
specific. MALS-24 in Hawaii should have different metrics for H-1 squadrons and CH- 
53 squadrons, for example. If the CH-53 fleet is encountering systematic supply issues, 
the quality of support for the H-1 squadron will not be negatively impacted. 

A MALS and T/M/S specific measurement system will result in a MALS 
reporting measurement for each T/M/S they support. As an example, MALS-24 will 
report a performance metric for their H-1 squadron, their CH-53 squadron, and their V-22 
squadron. When this delineation is made in the performance metric, MALS support for 
H-1 aircraft can be compared across all MALS that support H-1 aircraft. From a systems 
perspective, this gives AVLOG leaders an opportunity to identify where resources should 
be allocated to ensure flight line squadrons have what they need to succeed. 

J. CHAPTER SUMMARY 

Currently, the MALS examines itself according to several logistics metrics. No 
performance measurement system is currently used to evaluate the MALS performance 
as a whole and current metrics are insufficient to appropriate capture the output of the 
MALS. A proper performance measurement system must be a product of both 
maintenance and supply metrics that measure functions in a MALS. Returning to Caplice 
and Sheffi’s work (1995), deliveries to customers are the output of a logistics system and 
thus should be the focal point of a logistics performance measurement system. 

The metrics a MALS uses now adequately convey the performance of sub¬ 
functions within the organization. Optimizing each metric has different implications for 
the performance of the organization as a whole (in terms of support for flight line 
squadrons) and for the behavior of Marines in the MALS. Choosing any one of these 
metrics individually to be used as the measurement system for the MALS can have 
negative second- and third-order effects on the support a MALS provides to flight line 
squadrons. 

A MALS’s extent of control ceases at the point of delivery to a squadron. The 
goal of the MALS, therefore, cannot extent beyond this point. Increased aircraft 
readiness, therefore, cannot be the goal of a MALS. The MALSP MOD nodal logistics 
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system is strongly indicative of Marine Aviation’s intent to operate in a lean, efficient 
manner. Increasing inventory levels to support flight line squadrons is no longer part of 
the model and thus emphasizing metrics that encourage that type of behavior is to be 
discouraged. 
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V. CONCLUSION AND RECOMMENDATIONS 


A. CONCLUSION 

The MALS has many metrics available that measure its various sub-functions, 
though they vary in degrees of usefulness. For a metric to be a valuable logistic 
measurement, it needs to meet certain criteria. The predominant MALS metrics are 
evaluated against the criteria proposed by Caplice and Sheffi (1994) and summarized in 
Table 18. 


Table 18. Summary Comparison of Metrics and Evaluation Criteria 


Metric 

Criteria 

Buffer 
Status Red 
(BMT) 

TRR 

BCM 

Supply 

Effective¬ 

ness 

IMA 

Effective¬ 

ness 

SRT 

SCRT 

MALS 

Effective¬ 

ness 

Validity 

5 

3 

4 

4 

2 

3 

4 

3 

Robustness 

3 

3 

3 

4 

3 

3 

3 

2 

Usefulness 

4 

3 

4 

4 

3 

3 

5 

2 

Integration 

4 

3 

2 

4 

2 

2 

4 

2 

Economy 

4 

2 

3 

4 

3 

2 

4 

2 

Compatibility 

5 

5 

5 

5 

4 

5 

5 

4 

Level of Detail 

5 

3 

4 

4 

2 

3 

4 

2 

Behavioral 

Soundness 

4 

2 

3 

3 

3 

3 

4 

2 

Sum 

34 

24 

28 

32 

22 

24 

33 

19 


The most useful metrics (e.g., physical buffer status red, supply chain response 
time, supply effectiveness) sufficiently fulfill their purpose for measuring their respective 
function within the MALS. Even these metrics, however, largely fall short in measuring 
the overall output of the MALS and thus do not meet the standard for a performance 
measurement system. Eor a performance measurement system to be most beneficial to the 
organization, it needs to meet the criteria proposed by Caplice and Sheffi (1995) for a 
performance measurement system. Individual metrics are unable to be comprehensive or 
horizontally integrated (1995.). 
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1. Primary Questions 

(1) Of the metrics that MALS currently employs, which measures (or 
combination thereof) most appropriately reflect the level of support the 
MALS provides the flight-line? 

Of the metrics that MALS currently employs, Supply Effectiveness most 
appropriately reflects the level of support the MALS provides the flight-line. This metric 
most comprehensively measures the majority output of the MALS. Of the current 
metrics, it best captures the overall function of the MALS. Most sub-functions within the 
MALS contribute to this end metric in some manner. 

This metric is not necessarily ideal, however. It is not very descriptive in terms of 
what drives high or low support. In order to know how to improve Supply Effectiveness, 
leaders must examine separate metrics individually to determine the errant process. This 
metric gives leaders no indication of where to find the underlying problem. 

Eurthermore, this metric is framed positively, is inclusive of high- and low- 
priority requisitions, and provides no time domain. By constructing the metric in this 
manner, no indication is provided about the true level of support a MAES is providing. If 
Supply Effectiveness is 90%, it may be understood that a high level of support is being 
provided. If that 90%, however, is primarily low priority consumable requisitions and the 
remaining 10% are primarily NMCS requisitions that have remained off-station for 30 
days, then the true level of support may be actually quite low. This metric lacks sufficient 
detail to inform leaders about the true level of support provided to a flight-line squadron. 

(2) Of the metrics that MAES currently employs, which measures most 
appropriately incentivizes the most appropriate behavior to support the 
flight-line? 

Of the metrics that MAES currently employs. Repairable Physical Buffer Status 
Red and TRR-Supply incentivize the most appropriate behavior to support the flight-line. 
Physical buffer status red incentivizes maintenance and supply to both focus work efforts 
on those items which, if requisitioned by the flight-line, will not be able to be filled. 
Prioritizing these items contributes to filling these holes on the supply shelf so they are 
available when needed. TRR-Supply incentivizes proper forecasting, expediting, and 
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inventory management. A reliable TRR metric aids in proper inventory management and 
prioritization of expedited requisitions. By examining TRR more closely, supply is able 
to adjust inventory levels or expedite requisitions appropriately to increase supply 
effectiveness. 

(3) Are there metrics that MALS does not employ that should be adopted? 

Two metrics should be adopted by the MALS. First, the Supply Chain Response 
Time metric should be adopted as a logistic metric. Second, a weighted performance 
metric should be adopted as an overall performance system. 

First, Supply Chain Response Time should be adopted by the MALS, specifically 
in regard to NMCS and PMCS requisitions by T/M/S aircraft. By measuring the 
squadron’s total on-station and off-station wait time for NMCS and PMCS requisitions, a 
stronger signal is sent to the entire supply chain than by just measuring on-station wait 
time. While it is true that the MALS only controls the amount of time a part is on-station, 
it is also true that a MALS has influence on portions of the supply chain which it does not 
own. 

Capturing this time domain of the supply chain from the customer’s perspective is 
an excellent measure of customer satisfaction. Furthermore, by capturing this metric 
according to the various T/M/S a MALS supports, systematic issues are more easily 
identified. A MALS may have a low average for SCRT but, unless it is delineated by 
T/M/S, poor support for a particular T/M/S may be masked due to a misleading average. 

Second, a weighted performance metric should be adopted as a performance 
measurement system for each T/M/S a MALS supports. This is helpful in unifying the 
MALS under one common goal that is specific and measureable. The primary 
components of this performance metric should be: 

• Physical Buffer Status Red of Repairables: The percentage of repairable 
items carried that have a physical buffer status of red 

• Supply Chain Response Time: The percentage of NMCS and PMCS 
requisitions that went off-station with an SCRT exceeding a certain 
threshold 
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• Supply Effectiveness: The percentage of NMCS and PMCS requisitions 
that went off-station 

By presenting these metrics as percentages, they can more easily be compared to 
other MAES that support the same T/M/S. These metrics provide a holistic view of the 
output of a MAES and provide enough detail to MAES and AVEOG leadership to 
identify why support for a flight-line squadron may be poor. Condensing this 
performance metric to these three individual metrics enables easy communication of the 
metric. 

While these metrics do not completely capture the output from a MAES and all 
the services they perform for squadrons, they do summarize the majority of the output 
from maintenance and supply departments. Though the Repairable Physical Buffer Status 
Red metric is a measure of the health of the supply shelf, maintenance is primarily 
responsible for replenishing those repairable items. The SCRT metric captures the impact 
off-station items have on the flight-line. The Supply Effectiveness metric captures both 
consumable and repairable requisitions that went off-station. 

By combining these metrics into one measurement, there is a possibility that the 
end measurement may fail to identify problems at a MAES due to the flaw of averages. 
Eor example, a performance measurement goal of 85% can be reach if all three 
measurement score 85%. It can also be reached by averaging scores of 90%, 95%, and 
70%. By reporting only the end average, the lower 70% metric remains unidentified. To 
counteract this, each measurement should have an appropriate goal to define what 
success is and lower control limit to define what failure is. If a measurement falls below 
the lower control limit, it ought to be reported separately in order to identify the area for 
potential troubleshooting. 

To be meaningful, the SCRT metric needs to be measured in regard to a certain 
threshold. A threshold of three days, for example, measures the number of requisitions 
exceeding three days in customer wait time. Capturing this time domain is a strong 
indicator of responsiveness of the supply chain. 
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2. Secondary Question 

(1) To what extent can standardized performance measurements be used to 
measure flight-line support performance between various MALS? 

In order to measure and compare flight-line support performance between various 
MALS, the measurements need to first be T/M/S specific within each MALS. This 
accounts for the systematic differences between different fleets of aircraft. Legacy 
aircraft, upgraded aircraft, and new T/M/S all have different support systems. A MALS 
may have very limited repair capability on a new T/M/S due to manufacturer proprietary 
rights. Conversely, a MALS have a great deal of repair capability on decades-old legacy 
aircraft. 

If a MALS does not measure its performance according to T/M/S, then that 
MALS cannot be compared with other MALS. A MALS that supports primarily H-1 
aircraft cannot be directly compared with a MALS that supports primarily F-18 aircraft. 
The types of repairs required, availability of parts, and mission requirements for these 
different aircraft limit the ability to compare their supporting MALS on like terms. 

By measuring performance according to T/M/S, AVLOG leadership is better able 
to identify where it should allocate resources for process improvements. Systematic 
difficulties in the CH-53 community, for example, will be difficult to identify if those 
maintenance or supply issues are not captured separately. Poor support performance for a 
particular T/M/S may be disregarded if it is averaged in a conglomerate metric with all 
T/M/S. 

B. RECOMMENDATIONS 

The first recommendation is to include a time domain when measuring the 
performance of a MALS. A time domain is generally absent among the majority of 
metrics used within the MALS. Unfilled requisitions do hinder flight-line squadrons from 
receiving the parts that they need, but the true impact of unfilled requisitions cannot be 
understood without measuring how long squadrons waiting to receive those parts. A 
requisition for an NMCS discrepancy has very different implications for a squadron if it 
is delivered 30 days rather than three days. 
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The second recommendation is to ensure metrics are T/M/S specific. Resources 
may be misallocated if MALS are compared to one another without this distinction. By 
measuring a fleet of aircraft more holistically, a stronger signal is sent when a systematic 
problem occurs. 

The third recommendation is to not abandon logistic metrics of internal functions 
within a MALS when adopting a holistic performance metric for the MALS. The internal 
logistic metrics are the primary way in which leaders can identify why the MALS 
performance measurement may be low or what is causing it to succeed. Internally, 
departments and divisions should continue to measure their internal processes. These 
internal metrics, however, need to be understood in relation to the MALS performance 
metric and how they affect one another. 

Lastly, further research is recommended in two areas. First, cross validation of the 
scores given to individual metrics in this study should be conducted. Subject matter 
experts from the AVLOG community ought to independently score the metrics and then 
the inter-rater reliability or correlation between the scores can be measured. Second, 
appropriate weights ought to be determined for each metric used in the final performance 
metric. AVLOG leaders may determine if these metrics should be assigned equal weight 
or if any of them should receive more or less weight for final computation. Weights could 
also be assigned by analyzing the level of influence each metric’s performance has on 
total MALS output. This will prevent any of the measurements from unnecessarily 
skewing the measurement due to disproportionate allocation of weight. 
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